-- Logs begin at Sat 2022-08-06 00:26:30 UTC, end at Sat 2022-08-06 01:40:57 UTC. -- Aug 06 01:03:41.641105 dv-cinder-851996-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 06 01:03:41.649766 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 06 01:03:41.651753 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Aug 6 01:03:41 2022] *** Aug 06 01:03:41.651753 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 01:03:41.651753 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 01:03:41.651753 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: nodename: dv-cinder-851996-2-iscsi Aug 06 01:03:41.651753 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: machine: x86_64 Aug 06 01:03:41.651753 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: clock source: unix Aug 06 01:03:41.651753 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: pcre jit disabled Aug 06 01:03:41.651753 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: detected number of CPU cores: 4 Aug 06 01:03:41.651753 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: current working directory: / Aug 06 01:03:41.651753 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: detected binary path: /usr/bin/uwsgi-core Aug 06 01:03:41.652377 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: your processes number limit is 47680 Aug 06 01:03:41.652377 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: your memory page size is 4096 bytes Aug 06 01:03:41.652377 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: detected max file descriptor number: 2048 Aug 06 01:03:41.652377 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: lock engine: pthread robust mutexes Aug 06 01:03:41.652377 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: thunder lock: enabled Aug 06 01:03:41.652377 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 06 01:03:41.652377 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 06 01:03:41.693092 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: Python main interpreter initialized at 0x55574ea6b9c0 Aug 06 01:03:41.693092 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: python threads support enabled Aug 06 01:03:41.693092 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: your server socket listen backlog is limited to 100 connections Aug 06 01:03:41.693092 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: your mercy for graceful operations on workers is 90 seconds Aug 06 01:03:41.693487 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 01:03:41.693572 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: *** Operational MODE: preforking *** Aug 06 01:03:41.693800 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: *** uWSGI is running in multiple interpreter mode *** Aug 06 01:03:41.693800 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: spawned uWSGI master process (pid: 80518) Aug 06 01:03:41.694002 dv-cinder-851996-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 06 01:03:41.695817 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: spawned uWSGI worker 1 (pid: 80524, cores: 1) Aug 06 01:03:41.695817 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: spawned uWSGI worker 2 (pid: 80525, cores: 1) Aug 06 01:03:41.695817 dv-cinder-851996-2-iscsi devstack@keystone.service[80518]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 01:03:47.180822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 01:03:47.180822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: from cryptography.utils import int_from_bytes Aug 06 01:03:47.181602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 01:03:47.181602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: from cryptography.utils import int_from_bytes Aug 06 01:03:47.181726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 01:03:47.181726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: from cryptography.utils import int_from_bytes Aug 06 01:03:47.181726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 01:03:47.181726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: from cryptography.utils import int_from_bytes Aug 06 01:03:47.231062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 06 01:03:47.231254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 06 01:03:47.231429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 06 01:03:47.231527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] command line args: [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 06 01:03:47.231633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 06 01:03:47.231719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 06 01:03:47.231859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] command line args: [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 06 01:03:47.231957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ================================================================================ {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 06 01:03:47.232131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] admin_token = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.232217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 06 01:03:47.232318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ================================================================================ {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 06 01:03:47.232403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] config_dir = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.232550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.232639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] admin_token = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.232719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] config_source = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.232818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] config_dir = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.232904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] debug = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.233012 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.233102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.233314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] config_source = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.233377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] debug = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.233436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.233562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.233703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.233810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.233908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.234003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.234108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.234206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.234314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.234443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.234566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_config_append = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.234666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.234767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.234864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_dir = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.234973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_config_append = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.235071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_file = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.235166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.235279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.235361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_dir = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.235361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_file = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.235552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.235671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.235755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.235869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.235945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.236072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.236164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.236244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.236363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.236441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.236565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.236653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.236732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.236867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.236949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.237108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.237234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.237323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.237397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.237556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.237636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.237789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.237867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.238029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.238156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.238278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.238358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.238526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.238771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.238858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.238956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] publish_errors = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.239187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.239297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.239414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.239492 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] publish_errors = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.239603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.239734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.239858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.239969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.240097 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.240250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.240367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] standard_threads = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.240453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.240598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.240685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.240839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.240917 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] standard_threads = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.240998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.241076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.241176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] use_journal = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.241325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.241409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] use_json = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.241506 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.241594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] use_stderr = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.241667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] use_syslog = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.241737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] use_journal = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.241853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] use_json = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.241932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.242032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] use_stderr = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.242120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.242194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] use_syslog = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.242347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.242428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 01:03:47.242605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.242690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.242842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.242918 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.243074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.243160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.243310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.243390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.243557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.243637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.243799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.243881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.244058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.244148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.244304 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.244390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.244552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.244637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.244788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.244881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.245035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.245122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.245269 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.245348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.245515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.245602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.245757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.245837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.245988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.246073 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.246246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.246335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.246487 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.246572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.246725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.246809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.246959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.247049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.247212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.247292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.247450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.247536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.247707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.247834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.247977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.248057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.248210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.248290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.248440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.248606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.248683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.248754 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.248891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.248976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.249137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.249225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.249379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.249457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.249603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.249721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.249833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.249929 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.250078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.250163 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.250312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.250392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.250545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.250636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.250791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.250870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.251021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.251099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.251255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.251345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.251490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.251570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.251722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.251874 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.251977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.252090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.252189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.252286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.252412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.252554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.252666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.252748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.252860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.252939 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.253095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.253182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.253331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.253415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.253560 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.253644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.253794 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.253891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.254021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.254152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.254236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.254318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.254476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.254557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.254708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.254847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.254935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.255020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.255162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.255242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.255394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.255469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.255632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.255713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.255923 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.256016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.256169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.256258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.256403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.256482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.256641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.256721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.256873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.256961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.257112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.257192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.257349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.257437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.257576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.257655 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.257812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.257893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.258047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.258127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.258288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.258367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.258528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.258609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.258771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.258851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.259015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.259096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.259250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.259339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.259483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.259562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.259717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.259845 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.260007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.260085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.260211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.260293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.260447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.260526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.260689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.260770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.260924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.261003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.261158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.261249 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.261391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.261480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.261625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.261705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.261857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.261937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.262094 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.262180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.262335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.262423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.262571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.262654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.262806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.262900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.263045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.263124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.263279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.263360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.263512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.263600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.263746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.263873 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.264000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.264081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.264230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.264321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.264457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.264537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.264690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.264769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.264916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.265003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.265154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.265238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.265384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.265464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.265627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.265713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.265858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.265937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.266107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.266186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.266349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.266438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.266604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.266683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.266836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.external = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.266916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.267068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.267160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.267319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.external = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.267401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.267554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.267660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.267805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.password = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.267889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.268041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] auth.token = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.268134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.268280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.268361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.password = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.268514 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.268595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] auth.token = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.268759 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.268852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.268990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.269077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.269236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.269317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.269470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.269558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.269698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.269786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.269933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.270023 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.270177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.270267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.270415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.270559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.270660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.270750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.270894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.270982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.271133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.271215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.271890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.271981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.272155 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.272237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.272397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.272478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.272645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.272725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.272896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.272978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.273137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.273219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.273394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.273485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.273651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.273733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.273892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.273979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.274145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.274227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.274371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.274448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.274598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.274679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.274842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.274930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.275085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.275167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.275312 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.275393 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.275556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.275638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.275811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.275905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.276058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.276138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.276293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.276374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.276546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.276629 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.276783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.276863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.277025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.277105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.277259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.277342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.277506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.277587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.277744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.277824 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.277974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.278055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.278209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.278290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.278455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.278538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.278688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.278771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.278923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.279006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.279159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.279238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.279408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.279491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.279636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.279718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.279899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.279993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.280131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.280214 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.280373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.280454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.280605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.280685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.280850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.280932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.281100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.281180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.281341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.281430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.281598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.281678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.281823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.281901 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.282054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.282144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.282283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.282365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.282518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.282600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.282748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.282837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.282980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.283062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.283219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.283300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.283458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.283547 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.283700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.283836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.283955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.284037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.284208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.284300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.284444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.284526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.284676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.284755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.284922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.285011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.285162 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.285242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.285387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.285466 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.285621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.285710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.285849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.285931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.286084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.286165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.286317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.286409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.286552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.286632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.286793 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.286874 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.287085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.287718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.288022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.288284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.288518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.288749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.289019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.289343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.289582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.289893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.289972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.290172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.290316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.290460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.290738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.290809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.291171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.291451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.291612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.291721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.291890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.292155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.292415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.292670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.293068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.293501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.293818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.293903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.294067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.294194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.294320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.294454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.294581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.294772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.295027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.295281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.295447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.295647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.295720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.295931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.296004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.296171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.296240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.296424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.296497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.296672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.296742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.296913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.296984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.297152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.297252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.297389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.297517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.297635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.297769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.297887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.298013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.298126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.298452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.298530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.298658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.298729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.298902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.298972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.299138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.299207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.299374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.299446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.299637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.299707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.300070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.300351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.300492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.300577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.300735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.301005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.301145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.301271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.301405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.301539 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.301665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.301790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.301922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.302048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.302249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.302376 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.302501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.302501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.302810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.302937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.303071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.303198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.303324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.303449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.303574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.303701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.303936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.304064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] role.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.304194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.304323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] role.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.304451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.304588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.304723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.304852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.304981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.305107 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.305235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.305361 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.305494 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.305622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.305749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.305876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.306003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.306130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.306263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.306399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.306525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.306652 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.306777 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.306904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.307040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.307167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.307295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.307420 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.307548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.307675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.307832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.307960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.308107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.308236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.308365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.308493 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.308630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.308759 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.308887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.309014 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.309144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] role.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.309271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.309407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] role.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.309407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.309647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.309647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.309899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.310029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.310165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.310294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.310422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.310550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.310677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.310813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.310941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.311026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.311196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.311321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.311450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.311597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.311726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.311906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.312035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.312162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.312290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.312428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.312556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.312683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.312809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.312936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.313063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.313198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.313198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.313449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.313575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.313701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.313829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.313963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.314090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.314217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.314344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.314471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.314597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.314731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.314858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.314993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.315120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.315248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.315374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.315508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.315635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.315761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.315922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.316047 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.316173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.316308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.316435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.316560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.316684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.316817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.316947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.316947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.317195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.317195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.317436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.317436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.connection = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.317687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.317826 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.317959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.318093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.318218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.318359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.318484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.318622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.318768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.318893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.319018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.319144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.319270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.319399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.319536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.319664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.319816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.319943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.320069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.320193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.320345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.320473 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.320603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.320727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.320850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.320977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.321090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.321222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.321346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.321472 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.321596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.321724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.321859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.321982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.322118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.322247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.322247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 06 01:03:47.322490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.322490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.322490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.322490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.connection = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.322490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.322490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.323276 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.323401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.323534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.323774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.324309 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.324557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.324813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.325058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.325310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.327384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.327672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.327963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.328234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.328488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.328748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.329004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 01:03:47.329181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 06 01:03:47.354123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.354358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.354561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.359024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.359264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.359497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.362881 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.367954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.392279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.392509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.395430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.395670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.444941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.445473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.495623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.495882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.496255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.496497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.560812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.560997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.561128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.561268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.561396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.561523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.561665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.561794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.561927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.562059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.562191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.562331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.599934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.600072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.600157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.600305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.600395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.600540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.600626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.600770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.600853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.600977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.601061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.601188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.601272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.601401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.612173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.612564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.624843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.625079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.625164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.625307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.625391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.625533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.625621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.625749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.625841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.625963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.626047 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.626174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80524) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.626256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.626390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80525) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 01:03:47.643600 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.643837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 01:03:47.643979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.644176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.644261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 01:03:47.644381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 01:03:47.644523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.644655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.644740 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 01:03:47.644917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.644999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.645174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.645259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.645428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.645511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.645733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.655669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.655975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.656063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.656248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.656334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.656530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.656631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.656825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.663509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.663766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.668305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.668540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.668625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.668824 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.668911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.669112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.669203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.669402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.669485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.669673 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.669755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.669943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.670033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.670214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.670294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.670482 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.670563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.670752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.670834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.671025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.671113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.671295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.695120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.696608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.696899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.699885 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.701414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.701698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.703349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.703640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.703960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.706400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.706682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.707041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.713250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.713540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.714426 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.714737 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.717679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.717971 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.718273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.718560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.718807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.718889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.719093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.719177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.719376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.719460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.719654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.719829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.719979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.720267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.720548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.720818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.735699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.736004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.736605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.736882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.739966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.740822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.743159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.743461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.743983 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.744280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.747914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.748659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.751320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.752155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.754543 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.754850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.755143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.755292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.755432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.755591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.756264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.756428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.756550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.756715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.757030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.757308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.773180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.773467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.773738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.773852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.774007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.774137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.774405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.774671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.780915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.781570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.782494 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.782784 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.783114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.783388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.785649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.785937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.786233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.786316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.786511 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.786591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.786799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.786883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.787105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.787396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.798452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.798764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.798855 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.799051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.799138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.799349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.799435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.799631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.799747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.799936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.800078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.800209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.800402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.800495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.816026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.816229 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.819585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.819752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.822553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.822726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.825315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.825524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.826831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.827089 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.827222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.827388 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.827474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.827673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.833835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.834717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.836007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.836374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.838934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.839223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.839306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.839507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.839591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.839811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.839922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.840149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.847813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.848108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.848234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.848388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.848525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.848809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.855247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80524) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.855531 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.855753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80525) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 01:03:47.855862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.856061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.856143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.856352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.856433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.856621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.856708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.856888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.856973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.857161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.857253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.857436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.857519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.857721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.857805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.858070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.858153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.858335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.858418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.858601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.858693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.858872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80524) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.858954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.859152 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.859426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80525) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 01:03:47.892513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55574ea6b9c0 pid: 80524 (default app) Aug 06 01:03:47.893365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55574ea6b9c0 pid: 80525 (default app) Aug 06 01:03:47.895771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50080aff-e7a4-4298-b36d-4723f2161f53 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:03:47.896084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50080aff-e7a4-4298-b36d-4723f2161f53 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:03:47.896373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50080aff-e7a4-4298-b36d-4723f2161f53 None None] PATH_INFO: `/v3` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:03:47.901222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Sat Aug 6 01:03:47 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:03:56.186899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d25e10-fab5-4f2b-b8a6-72d1a82227bc None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:03:56.187425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d25e10-fab5-4f2b-b8a6-72d1a82227bc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:03:56.187425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d25e10-fab5-4f2b-b8a6-72d1a82227bc None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:03:56.191888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:03:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:03:56.196980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:03:56.197371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:03:56.197540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:03:56.269211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: INFO dbcounter [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] Registered counter for database keystone Aug 06 01:03:56.289106 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG oslo_db.sqlalchemy.engines [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80524) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 06 01:03:56.295357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writer thread running {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 06 01:03:56.470283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG passlib.handlers.bcrypt [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80524) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 06 01:03:56.470698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG passlib.handlers.bcrypt [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80524) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 06 01:03:56.524623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:03:56.526212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:03:56.654227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:03:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 06 01:03:56.794069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d25e10-fab5-4f2b-b8a6-72d1a82227bc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:03:56.865504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: INFO dbcounter [None req-e4d25e10-fab5-4f2b-b8a6-72d1a82227bc None None] Registered counter for database keystone Aug 06 01:03:56.886838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG oslo_db.sqlalchemy.engines [None req-e4d25e10-fab5-4f2b-b8a6-72d1a82227bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80525) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 06 01:03:56.893010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writer thread running {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 06 01:03:57.014570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d25e10-fab5-4f2b-b8a6-72d1a82227bc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:03:57.018892 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bca7b-7c37-4212-a34c-b41b07e9c693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:03:57.019534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bca7b-7c37-4212-a34c-b41b07e9c693 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:03:57.019840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bca7b-7c37-4212-a34c-b41b07e9c693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:03:57.020117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bca7b-7c37-4212-a34c-b41b07e9c693 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:03:57.028213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bca7b-7c37-4212-a34c-b41b07e9c693 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:03:57.080083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d90bca7b-7c37-4212-a34c-b41b07e9c693 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 06 01:03:57.080373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bca7b-7c37-4212-a34c-b41b07e9c693 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:03:57.090031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Sat Aug 6 01:03:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:03:57.096070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:03:57.110469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38a813-e327-431c-ba90-2a045b1ade7c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:03:57.115631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6ada9-fb3c-497d-99de-97ca9b992f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:03:57.116799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6ada9-fb3c-497d-99de-97ca9b992f5d admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:03:57.117090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6ada9-fb3c-497d-99de-97ca9b992f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:03:57.117368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6ada9-fb3c-497d-99de-97ca9b992f5d admin admin] PATH_INFO: `/v3/projects/58e81a1daacf4bb08ea62cf133cf1073` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:03:57.118931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6ada9-fb3c-497d-99de-97ca9b992f5d admin admin] RBAC: Authorizing `identity:get_project(project_id=58e81a1daacf4bb08ea62cf133cf1073)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:03:57.168500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WARNING keystone.common.rbac_enforcer.enforcer [None req-18d6ada9-fb3c-497d-99de-97ca9b992f5d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 06 01:03:57.168848 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6ada9-fb3c-497d-99de-97ca9b992f5d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:03:57.173502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Sat Aug 6 01:03:57 2022] GET /identity/v3/projects/58e81a1daacf4bb08ea62cf133cf1073 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:03:59.141356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149375e-c366-4778-b799-f3b6e5d518da None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:03:59.141864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149375e-c366-4778-b799-f3b6e5d518da None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:03:59.141951 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149375e-c366-4778-b799-f3b6e5d518da None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:03:59.146006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:03:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:03:59.151854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73498-8efa-4b61-9e3c-310b82445d89 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:03:59.152210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73498-8efa-4b61-9e3c-310b82445d89 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:03:59.152420 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e73498-8efa-4b61-9e3c-310b82445d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:03:59.171766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e2e73498-8efa-4b61-9e3c-310b82445d89 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:03:59.173302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e2e73498-8efa-4b61-9e3c-310b82445d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:03:59.181095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:03:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 06 01:03:59.202982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149375e-c366-4778-b799-f3b6e5d518da None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:03:59.215257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149375e-c366-4778-b799-f3b6e5d518da None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:03:59.219341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe39afc-d146-457a-b5a9-364fdd7e2fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:03:59.219989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe39afc-d146-457a-b5a9-364fdd7e2fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:03:59.220270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe39afc-d146-457a-b5a9-364fdd7e2fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:03:59.220545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe39afc-d146-457a-b5a9-364fdd7e2fc3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:03:59.228056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe39afc-d146-457a-b5a9-364fdd7e2fc3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:03:59.229817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe39afc-d146-457a-b5a9-364fdd7e2fc3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:03:59.238355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Sat Aug 6 01:03:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 01:03:59.243984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e73498-8efa-4b61-9e3c-310b82445d89 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:03:59.252729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e73498-8efa-4b61-9e3c-310b82445d89 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:03:59.257630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5fc5e-140f-4c3f-a8fb-ea798c82adc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:03:59.258251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5fc5e-140f-4c3f-a8fb-ea798c82adc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:03:59.258537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5fc5e-140f-4c3f-a8fb-ea798c82adc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:03:59.258815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5fc5e-140f-4c3f-a8fb-ea798c82adc2 admin admin] PATH_INFO: `/v3/users/bb37d398d5fd456aa46fd2b5f5576815` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:03:59.260197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5fc5e-140f-4c3f-a8fb-ea798c82adc2 admin admin] RBAC: Authorizing `identity:get_user(user_id=bb37d398d5fd456aa46fd2b5f5576815)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:03:59.262021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5fc5e-140f-4c3f-a8fb-ea798c82adc2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:03:59.263556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Sat Aug 6 01:03:59 2022] GET /identity/v3/users/bb37d398d5fd456aa46fd2b5f5576815 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:01.287927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737066f-9d66-4990-aacc-30acfa159ff7 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:01.288499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737066f-9d66-4990-aacc-30acfa159ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:01.288499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737066f-9d66-4990-aacc-30acfa159ff7 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:01.289831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:01.294963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bfcf6-e41c-4db7-88a1-c70d911a3287 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:01.295370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bfcf6-e41c-4db7-88a1-c70d911a3287 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:01.295526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bfcf6-e41c-4db7-88a1-c70d911a3287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:01.314912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6a4bfcf6-e41c-4db7-88a1-c70d911a3287 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:01.316140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6a4bfcf6-e41c-4db7-88a1-c70d911a3287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:01.325195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823d35e-4ab6-437d-9cb2-b977f59c6769 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:01.325490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823d35e-4ab6-437d-9cb2-b977f59c6769 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:01.325772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823d35e-4ab6-437d-9cb2-b977f59c6769 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:01.328366 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:01.333723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a65453-d891-4923-b927-dc454bd67a9a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:01.334029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a65453-d891-4923-b927-dc454bd67a9a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:01.334305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a65453-d891-4923-b927-dc454bd67a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:01.379909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG passlib.handlers.bcrypt [None req-d2a65453-d891-4923-b927-dc454bd67a9a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80525) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 06 01:04:01.380432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG passlib.handlers.bcrypt [None req-d2a65453-d891-4923-b927-dc454bd67a9a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80525) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 06 01:04:01.410002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:01.426040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d2a65453-d891-4923-b927-dc454bd67a9a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:01.427612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d2a65453-d891-4923-b927-dc454bd67a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:01.433000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bfcf6-e41c-4db7-88a1-c70d911a3287 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:01.456347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:01.527579 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:01.532699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bfcf6-e41c-4db7-88a1-c70d911a3287 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:01.549508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a65453-d891-4923-b927-dc454bd67a9a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:01.565992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dc3c7-e3f6-45aa-9d8d-415f7125fd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:01.566640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dc3c7-e3f6-45aa-9d8d-415f7125fd44 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:01.566928 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dc3c7-e3f6-45aa-9d8d-415f7125fd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:01.567205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dc3c7-e3f6-45aa-9d8d-415f7125fd44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:01.578713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dc3c7-e3f6-45aa-9d8d-415f7125fd44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-3b5dc3c7-e3f6-45aa-9d8d-415f7125fd44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:01.588290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:01.589447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:01.590317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:01.595140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dc3c7-e3f6-45aa-9d8d-415f7125fd44 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:01.651085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a65453-d891-4923-b927-dc454bd67a9a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:01.653335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:01.677909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:01.687581 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481068b-c9f9-4611-a24f-f3dee22c792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:01.688579 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481068b-c9f9-4611-a24f-f3dee22c792f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:01.688865 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481068b-c9f9-4611-a24f-f3dee22c792f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:01.689143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481068b-c9f9-4611-a24f-f3dee22c792f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:01.695251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dc3c7-e3f6-45aa-9d8d-415f7125fd44 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:01.729456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4f030-067b-4a7e-aebf-f1daddedf4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:01.730088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4f030-067b-4a7e-aebf-f1daddedf4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:01.730374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4f030-067b-4a7e-aebf-f1daddedf4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:01.730652 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4f030-067b-4a7e-aebf-f1daddedf4a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:01.731399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4f030-067b-4a7e-aebf-f1daddedf4a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:01.731709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4f030-067b-4a7e-aebf-f1daddedf4a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:01.733147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4f030-067b-4a7e-aebf-f1daddedf4a6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:01.744800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:01.750358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4f030-067b-4a7e-aebf-f1daddedf4a6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:01.756082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481068b-c9f9-4611-a24f-f3dee22c792f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:01.757593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481068b-c9f9-4611-a24f-f3dee22c792f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:01.848416 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4f030-067b-4a7e-aebf-f1daddedf4a6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:01.854371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 01:04:01.860323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481068b-c9f9-4611-a24f-f3dee22c792f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:01.863478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:01.964226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481068b-c9f9-4611-a24f-f3dee22c792f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:01.988026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a39765-e9a8-4629-9a33-a8e1a7e408e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:01.988685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a39765-e9a8-4629-9a33-a8e1a7e408e3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:01.988981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a39765-e9a8-4629-9a33-a8e1a7e408e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:01.989255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a39765-e9a8-4629-9a33-a8e1a7e408e3 None admin] PATH_INFO: `/v3/users/bb37d398d5fd456aa46fd2b5f5576815` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:01.990472 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a39765-e9a8-4629-9a33-a8e1a7e408e3 None admin] RBAC: Authorizing `identity:get_user(user_id=bb37d398d5fd456aa46fd2b5f5576815)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:01.991977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a39765-e9a8-4629-9a33-a8e1a7e408e3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:01.993577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:01 2022] GET /identity/v3/users/bb37d398d5fd456aa46fd2b5f5576815 => generated 247 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:01.998983 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a39765-e9a8-4629-9a33-a8e1a7e408e3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:02.003237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b029ca5-e560-43e7-ae83-e70308c99da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:02.004143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029ca5-e560-43e7-ae83-e70308c99da4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:02.004427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029ca5-e560-43e7-ae83-e70308c99da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:02.004700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029ca5-e560-43e7-ae83-e70308c99da4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:02.014315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:02.023761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b029ca5-e560-43e7-ae83-e70308c99da4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:02.025285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b029ca5-e560-43e7-ae83-e70308c99da4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-5b029ca5-e560-43e7-ae83-e70308c99da4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:04:02.042937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:02.043810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:02.044657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:04:02.044657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:04:02.044657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:04:02.044657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:02.044657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 9/18] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:02.049916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b029ca5-e560-43e7-ae83-e70308c99da4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:02.098005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a39765-e9a8-4629-9a33-a8e1a7e408e3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:02.105194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:02.129947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad431f-d33c-471a-a646-1ec0b5b98461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:02.130591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad431f-d33c-471a-a646-1ec0b5b98461 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:02.131404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad431f-d33c-471a-a646-1ec0b5b98461 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:02.131679 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad431f-d33c-471a-a646-1ec0b5b98461 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:02.132889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad431f-d33c-471a-a646-1ec0b5b98461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:02.134317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad431f-d33c-471a-a646-1ec0b5b98461 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:02.135993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 10/19] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:02.141644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad431f-d33c-471a-a646-1ec0b5b98461 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:02.148879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b029ca5-e560-43e7-ae83-e70308c99da4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:02.180886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77ef2e-b68a-403d-9f5f-39ae880a49ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:02.181503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77ef2e-b68a-403d-9f5f-39ae880a49ed None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:02.181791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77ef2e-b68a-403d-9f5f-39ae880a49ed None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:02.182077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77ef2e-b68a-403d-9f5f-39ae880a49ed None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:02.182777 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77ef2e-b68a-403d-9f5f-39ae880a49ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:02.183085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77ef2e-b68a-403d-9f5f-39ae880a49ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:02.184502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77ef2e-b68a-403d-9f5f-39ae880a49ed None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:02.202230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 10/20] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:02.223243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:02.243884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad431f-d33c-471a-a646-1ec0b5b98461 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:02.280405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69afe10a-9e69-4bec-b154-b569e37248e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:02.281056 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afe10a-9e69-4bec-b154-b569e37248e6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:02.281345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afe10a-9e69-4bec-b154-b569e37248e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:02.281624 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afe10a-9e69-4bec-b154-b569e37248e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:02.282445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69afe10a-9e69-4bec-b154-b569e37248e6 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.domain.id=default, user.id=bb37d398d5fd456aa46fd2b5f5576815) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:02.282761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69afe10a-9e69-4bec-b154-b569e37248e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:02.284294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69afe10a-9e69-4bec-b154-b569e37248e6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:02.291850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 11/21] 10.222.0.25 () {56 vars in 1322 bytes} [Sat Aug 6 01:04:02 2022] GET /identity/v3/role_assignments?user.id=bb37d398d5fd456aa46fd2b5f5576815&scope.domain.id=default&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:04.765195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d25f22-52fc-46b1-bd5e-bd60b10cd3d3 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:04.765686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d25f22-52fc-46b1-bd5e-bd60b10cd3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:04.765785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d25f22-52fc-46b1-bd5e-bd60b10cd3d3 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:04.767285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 11/22] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:04.785371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f9c68-6670-4035-be6d-b2cec358156d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:04.786300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f9c68-6670-4035-be6d-b2cec358156d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:04.786788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f9c68-6670-4035-be6d-b2cec358156d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:04.792279 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0b54a-51b8-47f1-968e-81f625205bfb None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:04.792844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0b54a-51b8-47f1-968e-81f625205bfb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:04.793348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0b54a-51b8-47f1-968e-81f625205bfb None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:04.794876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 12/23] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:04.810683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938fb51-b957-4502-91a7-dc65e545bcc8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:04.811291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938fb51-b957-4502-91a7-dc65e545bcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:04.811771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938fb51-b957-4502-91a7-dc65e545bcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:04.813193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3f2f9c68-6670-4035-be6d-b2cec358156d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:04.814585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3f2f9c68-6670-4035-be6d-b2cec358156d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:04.845533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d938fb51-b957-4502-91a7-dc65e545bcc8 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:04.847075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d938fb51-b957-4502-91a7-dc65e545bcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:04.914547 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 12/24] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:04.916265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab421ff-1ef8-40ce-a762-5eb3b3b8cad7 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:04.916770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab421ff-1ef8-40ce-a762-5eb3b3b8cad7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:04.917250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab421ff-1ef8-40ce-a762-5eb3b3b8cad7 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:04.918699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 13/25] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:04.929127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312489-bfff-40b6-b414-416c213f4838 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:04.929423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312489-bfff-40b6-b414-416c213f4838 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:04.929714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312489-bfff-40b6-b414-416c213f4838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:04.959608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 13/26] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:04.961288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-36312489-bfff-40b6-b414-416c213f4838 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:04.962484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-36312489-bfff-40b6-b414-416c213f4838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:04.964509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938fb51-b957-4502-91a7-dc65e545bcc8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:05.047229 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:05.076467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 14/27] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:05.083799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36312489-bfff-40b6-b414-416c213f4838 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:05.087438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:05.120498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938fb51-b957-4502-91a7-dc65e545bcc8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:05.162140 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f9fda2-a566-4ba8-bb4b-d67780a6dae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:05.162786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9fda2-a566-4ba8-bb4b-d67780a6dae8 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:05.163075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9fda2-a566-4ba8-bb4b-d67780a6dae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:05.163357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9fda2-a566-4ba8-bb4b-d67780a6dae8 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:05.164122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9fda2-a566-4ba8-bb4b-d67780a6dae8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:05.165835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f9fda2-a566-4ba8-bb4b-d67780a6dae8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:05.212198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36312489-bfff-40b6-b414-416c213f4838 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:05.246618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e061d-35d7-4f94-9b18-28212b054369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:05.247375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e061d-35d7-4f94-9b18-28212b054369 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:05.247588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e061d-35d7-4f94-9b18-28212b054369 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:05.247870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e061d-35d7-4f94-9b18-28212b054369 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:05.248585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e061d-35d7-4f94-9b18-28212b054369 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:05.249978 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e061d-35d7-4f94-9b18-28212b054369 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:05.438257 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 15/28] 10.222.0.25 () {60 vars in 1131 bytes} [Sat Aug 6 01:04:05 2022] POST /identity/v3/roles => generated 219 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:05.439901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b61f95-4fc2-4df1-bb67-ab3cd0887a44 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:05.440256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b61f95-4fc2-4df1-bb67-ab3cd0887a44 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:05.440541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b61f95-4fc2-4df1-bb67-ab3cd0887a44 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:05.442247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 16/29] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:05.442991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b61f95-4fc2-4df1-bb67-ab3cd0887a44 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:05.489759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 14/30] 10.222.0.25 () {60 vars in 1131 bytes} [Sat Aug 6 01:04:04 2022] POST /identity/v3/roles => generated 225 bytes in 530 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:05.492280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33826a26-f49a-4872-be48-ed7138bce5fa None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:05.492856 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33826a26-f49a-4872-be48-ed7138bce5fa None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:05.493378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33826a26-f49a-4872-be48-ed7138bce5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:05.521849 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-33826a26-f49a-4872-be48-ed7138bce5fa None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:05.525643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-33826a26-f49a-4872-be48-ed7138bce5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:05.644987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:05.661064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 15/31] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:05.699700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b61f95-4fc2-4df1-bb67-ab3cd0887a44 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:05.707819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33826a26-f49a-4872-be48-ed7138bce5fa None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:05.737245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6061ab3-f961-4140-afec-a875a1838820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:05.737930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6061ab3-f961-4140-afec-a875a1838820 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:05.738218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6061ab3-f961-4140-afec-a875a1838820 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:05.738504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6061ab3-f961-4140-afec-a875a1838820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:05.751025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6061ab3-f961-4140-afec-a875a1838820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:05.752901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6061ab3-f961-4140-afec-a875a1838820 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:05.760266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e6061ab3-f961-4140-afec-a875a1838820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:04:05.775111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:05.776005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:05.776837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:04:05.776837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:04:05.776837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:04:05.776837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:05.777471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 17/32] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:05.788009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6061ab3-f961-4140-afec-a875a1838820 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:05.851850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33826a26-f49a-4872-be48-ed7138bce5fa None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:05.895870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db3b3f-605a-44c6-823b-77f0c372b5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:05.897144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db3b3f-605a-44c6-823b-77f0c372b5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:05.897436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db3b3f-605a-44c6-823b-77f0c372b5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:05.897714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db3b3f-605a-44c6-823b-77f0c372b5e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:05.913295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db3b3f-605a-44c6-823b-77f0c372b5e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-60db3b3f-605a-44c6-823b-77f0c372b5e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:05.920483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:05.921347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:05.922096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:05.922682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6061ab3-f961-4140-afec-a875a1838820 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:05.924781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 16/33] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:05.928538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0e501-9ad3-45cd-907d-76a8a2fb3f81 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:05.928538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0e501-9ad3-45cd-907d-76a8a2fb3f81 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:05.928538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0e501-9ad3-45cd-907d-76a8a2fb3f81 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:05.929006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 17/34] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:05.930877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0e501-9ad3-45cd-907d-76a8a2fb3f81 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:05.944139 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:05.965298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae1cbc-8279-4df8-927f-5b3d6e5e020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:05.966010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae1cbc-8279-4df8-927f-5b3d6e5e020a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:05.966294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae1cbc-8279-4df8-927f-5b3d6e5e020a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:05.966571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae1cbc-8279-4df8-927f-5b3d6e5e020a None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:05.967274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae1cbc-8279-4df8-927f-5b3d6e5e020a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:05.967585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae1cbc-8279-4df8-927f-5b3d6e5e020a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:05.969041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae1cbc-8279-4df8-927f-5b3d6e5e020a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:05.989053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 18/35] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:05.990639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861e3e0-fdb7-4486-9f1e-95e28c7eaaff None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:05.990936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861e3e0-fdb7-4486-9f1e-95e28c7eaaff None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:05.991219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861e3e0-fdb7-4486-9f1e-95e28c7eaaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.010662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0861e3e0-fdb7-4486-9f1e-95e28c7eaaff None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:06.012241 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0861e3e0-fdb7-4486-9f1e-95e28c7eaaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:06.034691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:06.043147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0e501-9ad3-45cd-907d-76a8a2fb3f81 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:06.077777 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2d55-75fa-479f-823b-fb2498226f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:06.078417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2d55-75fa-479f-823b-fb2498226f82 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:06.078706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2d55-75fa-479f-823b-fb2498226f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:06.078980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2d55-75fa-479f-823b-fb2498226f82 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.079712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c2d55-75fa-479f-823b-fb2498226f82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:06.080057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c2d55-75fa-479f-823b-fb2498226f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:06.081581 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c2d55-75fa-479f-823b-fb2498226f82 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:06.091804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 19/36] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:06.091804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861e3e0-fdb7-4486-9f1e-95e28c7eaaff None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:06.094265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 18/37] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:06.100377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2d55-75fa-479f-823b-fb2498226f82 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:06.200632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861e3e0-fdb7-4486-9f1e-95e28c7eaaff None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:06.203952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2d55-75fa-479f-823b-fb2498226f82 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:06.209948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:06.241672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e18b0-59e6-47a2-b514-89d105d17ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:06.241841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127a9c8-1d77-4a06-95cb-243f3663128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:06.242231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e18b0-59e6-47a2-b514-89d105d17ffc None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:06.242327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127a9c8-1d77-4a06-95cb-243f3663128d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:06.242517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e18b0-59e6-47a2-b514-89d105d17ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:06.242608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127a9c8-1d77-4a06-95cb-243f3663128d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:06.242795 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127a9c8-1d77-4a06-95cb-243f3663128d None admin] PATH_INFO: `/v3/users/bb37d398d5fd456aa46fd2b5f5576815` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.242884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e18b0-59e6-47a2-b514-89d105d17ffc None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.243483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e18b0-59e6-47a2-b514-89d105d17ffc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:06.244484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127a9c8-1d77-4a06-95cb-243f3663128d None admin] RBAC: Authorizing `identity:get_user(user_id=bb37d398d5fd456aa46fd2b5f5576815)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:06.244985 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e18b0-59e6-47a2-b514-89d105d17ffc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:06.245931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127a9c8-1d77-4a06-95cb-243f3663128d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:06.252021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 19/38] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:06 2022] GET /identity/v3/users/bb37d398d5fd456aa46fd2b5f5576815 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:06.252021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127a9c8-1d77-4a06-95cb-243f3663128d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:06.280299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:06.343605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 20/39] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:04:06 2022] POST /identity/v3/projects => generated 301 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:06.344475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e18b0-59e6-47a2-b514-89d105d17ffc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:06.360562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127a9c8-1d77-4a06-95cb-243f3663128d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:06.411144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c18297-ced9-40b0-ac09-e67004aec0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:06.411827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c18297-ced9-40b0-ac09-e67004aec0da None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:06.412145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c18297-ced9-40b0-ac09-e67004aec0da None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:06.412427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c18297-ced9-40b0-ac09-e67004aec0da None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.413171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c18297-ced9-40b0-ac09-e67004aec0da None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:06.414616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c18297-ced9-40b0-ac09-e67004aec0da None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:06.461180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e18b0-59e6-47a2-b514-89d105d17ffc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:06.495441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf11ad4-52c3-4838-81ac-1ea0355f818a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:06.496046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11ad4-52c3-4838-81ac-1ea0355f818a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:06.496334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11ad4-52c3-4838-81ac-1ea0355f818a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:06.496611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11ad4-52c3-4838-81ac-1ea0355f818a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.497975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf11ad4-52c3-4838-81ac-1ea0355f818a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:06.499493 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf11ad4-52c3-4838-81ac-1ea0355f818a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:06.501196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 21/40] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:06.502395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e525d36-552f-4fe2-9588-40d4f2e58c2d None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:06.502684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e525d36-552f-4fe2-9588-40d4f2e58c2d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:06.502958 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e525d36-552f-4fe2-9588-40d4f2e58c2d None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.504392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 22/41] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:06.509587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e525d36-552f-4fe2-9588-40d4f2e58c2d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:06.515906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 20/42] 10.222.0.25 () {60 vars in 1131 bytes} [Sat Aug 6 01:04:06 2022] POST /identity/v3/roles => generated 223 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:06.515906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cbf55-80c3-4216-8816-03245f696f2e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:06.515906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cbf55-80c3-4216-8816-03245f696f2e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:06.515906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8cbf55-80c3-4216-8816-03245f696f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.536825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fd8cbf55-80c3-4216-8816-03245f696f2e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:06.538482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fd8cbf55-80c3-4216-8816-03245f696f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:06.544607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:06.637162 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 21/43] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:06.654625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e525d36-552f-4fe2-9588-40d4f2e58c2d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:06.660049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8cbf55-80c3-4216-8816-03245f696f2e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:06.680099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:06.695581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966eac75-bbf2-4e79-a0c7-30b04a141d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:06.696349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966eac75-bbf2-4e79-a0c7-30b04a141d9c None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:06.696656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966eac75-bbf2-4e79-a0c7-30b04a141d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:06.696913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966eac75-bbf2-4e79-a0c7-30b04a141d9c None admin] PATH_INFO: `/v3/domains/default/users/bb37d398d5fd456aa46fd2b5f5576815/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.698972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966eac75-bbf2-4e79-a0c7-30b04a141d9c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bb37d398d5fd456aa46fd2b5f5576815, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:06.700726 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966eac75-bbf2-4e79-a0c7-30b04a141d9c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:06.765298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8cbf55-80c3-4216-8816-03245f696f2e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:06.785887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 23/44] 10.222.0.25 () {58 vars in 1361 bytes} [Sat Aug 6 01:04:06 2022] PUT /identity/v3/domains/default/users/bb37d398d5fd456aa46fd2b5f5576815/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:06.806302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e17a8-a6b2-435e-b164-cc09147c11b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:06.806977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e17a8-a6b2-435e-b164-cc09147c11b0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:06.807259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e17a8-a6b2-435e-b164-cc09147c11b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:06.807542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e17a8-a6b2-435e-b164-cc09147c11b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.808798 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e17a8-a6b2-435e-b164-cc09147c11b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:06.810362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e17a8-a6b2-435e-b164-cc09147c11b0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:06.813196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 22/45] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:06.819292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966eac75-bbf2-4e79-a0c7-30b04a141d9c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:06.910459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:06.925711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966eac75-bbf2-4e79-a0c7-30b04a141d9c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:06.963216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409a323-fbe0-4eab-a30f-cf903c2700e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:06.963972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409a323-fbe0-4eab-a30f-cf903c2700e9 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:06.964258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409a323-fbe0-4eab-a30f-cf903c2700e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:06.964537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409a323-fbe0-4eab-a30f-cf903c2700e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:06.965262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409a323-fbe0-4eab-a30f-cf903c2700e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:06.966721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409a323-fbe0-4eab-a30f-cf903c2700e9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:07.092730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:04:06 2022] POST /identity/v3/projects => generated 312 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:08.798564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3d60c-5302-4691-a6d4-f17a525c912a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:08.799063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3d60c-5302-4691-a6d4-f17a525c912a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:08.799146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3d60c-5302-4691-a6d4-f17a525c912a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:08.800504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 23/47] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:08.805728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400abf58-e29d-4ee7-898d-da7691033469 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:08.806143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400abf58-e29d-4ee7-898d-da7691033469 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:08.806367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400abf58-e29d-4ee7-898d-da7691033469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:08.824456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-400abf58-e29d-4ee7-898d-da7691033469 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:08.825898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-400abf58-e29d-4ee7-898d-da7691033469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:08.890285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 25/48] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:08.913315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3d60c-5302-4691-a6d4-f17a525c912a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:08.997797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:09.014269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3d60c-5302-4691-a6d4-f17a525c912a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:09.046829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171f4b-2184-45ac-b961-990bb2036f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:09.047469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171f4b-2184-45ac-b961-990bb2036f84 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:09.047756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171f4b-2184-45ac-b961-990bb2036f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:09.048064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171f4b-2184-45ac-b961-990bb2036f84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:09.058497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a171f4b-2184-45ac-b961-990bb2036f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8a171f4b-2184-45ac-b961-990bb2036f84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:09.063443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:09.064322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:09.065076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 24/49] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:09.071029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171f4b-2184-45ac-b961-990bb2036f84 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:09.071534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3add8-e5de-40a0-9b29-ea14905ac88a None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:09.071873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3add8-e5de-40a0-9b29-ea14905ac88a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:09.072184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3add8-e5de-40a0-9b29-ea14905ac88a None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:09.073537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 26/50] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:09.078601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f5a4-fa34-46a6-b581-640eec3b1c57 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:09.078891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f5a4-fa34-46a6-b581-640eec3b1c57 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:09.079173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f5a4-fa34-46a6-b581-640eec3b1c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:09.097449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8aa6f5a4-fa34-46a6-b581-640eec3b1c57 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:09.098765 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8aa6f5a4-fa34-46a6-b581-640eec3b1c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:09.158028 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:09.163466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 27/51] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:09.169083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171f4b-2184-45ac-b961-990bb2036f84 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:09.186398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f5a4-fa34-46a6-b581-640eec3b1c57 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:09.203846 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689743e-1e68-4020-a868-3ca04bddce27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:09.204472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689743e-1e68-4020-a868-3ca04bddce27 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:09.204763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689743e-1e68-4020-a868-3ca04bddce27 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:09.205045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689743e-1e68-4020-a868-3ca04bddce27 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:09.205765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689743e-1e68-4020-a868-3ca04bddce27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:09.206078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689743e-1e68-4020-a868-3ca04bddce27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:09.207476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689743e-1e68-4020-a868-3ca04bddce27 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:09.215437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 25/52] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:09.221046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689743e-1e68-4020-a868-3ca04bddce27 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:09.229510 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:09.284133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f5a4-fa34-46a6-b581-640eec3b1c57 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:09.317683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc71972-a7e8-4401-8027-fdd68a7b69f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:09.318892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc71972-a7e8-4401-8027-fdd68a7b69f2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:09.319188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc71972-a7e8-4401-8027-fdd68a7b69f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:09.319473 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc71972-a7e8-4401-8027-fdd68a7b69f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:09.319714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689743e-1e68-4020-a868-3ca04bddce27 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:09.320673 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc71972-a7e8-4401-8027-fdd68a7b69f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:09.322156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc71972-a7e8-4401-8027-fdd68a7b69f2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:09.323745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 28/53] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:09.329706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc71972-a7e8-4401-8027-fdd68a7b69f2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:09.353329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c6c7d-f6ee-4500-8078-6bf8cae94d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:09.353968 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c6c7d-f6ee-4500-8078-6bf8cae94d54 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:09.354256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c6c7d-f6ee-4500-8078-6bf8cae94d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:09.354533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c6c7d-f6ee-4500-8078-6bf8cae94d54 None admin] PATH_INFO: `/v3/users/bb37d398d5fd456aa46fd2b5f5576815` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:09.355732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c6c7d-f6ee-4500-8078-6bf8cae94d54 None admin] RBAC: Authorizing `identity:get_user(user_id=bb37d398d5fd456aa46fd2b5f5576815)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:09.357193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c6c7d-f6ee-4500-8078-6bf8cae94d54 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:09.358773 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 26/54] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:09 2022] GET /identity/v3/users/bb37d398d5fd456aa46fd2b5f5576815 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:09.364178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c6c7d-f6ee-4500-8078-6bf8cae94d54 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:09.399762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:09.434731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc71972-a7e8-4401-8027-fdd68a7b69f2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:09.444255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:09.467614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd044a26-15dd-4bb2-9702-c69ff32ca075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:09.468267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd044a26-15dd-4bb2-9702-c69ff32ca075 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:09.468551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd044a26-15dd-4bb2-9702-c69ff32ca075 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:09.468751 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c6c7d-f6ee-4500-8078-6bf8cae94d54 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:09.468839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd044a26-15dd-4bb2-9702-c69ff32ca075 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:09.469490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd044a26-15dd-4bb2-9702-c69ff32ca075 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:09.470953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd044a26-15dd-4bb2-9702-c69ff32ca075 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:09.503251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5c8a5-871c-4b6c-acb1-94058194b305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:09.503924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5c8a5-871c-4b6c-acb1-94058194b305 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:09.504217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5c8a5-871c-4b6c-acb1-94058194b305 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:09.504503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5c8a5-871c-4b6c-acb1-94058194b305 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:09.505671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5c8a5-871c-4b6c-acb1-94058194b305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:09.507209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d5c8a5-871c-4b6c-acb1-94058194b305 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:09.508828 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 27/55] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:09.514469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5c8a5-871c-4b6c-acb1-94058194b305 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:09.573125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 29/56] 10.222.0.25 () {60 vars in 1140 bytes} [Sat Aug 6 01:04:09 2022] POST /identity/v3/projects => generated 298 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:04:09.624265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d5c8a5-871c-4b6c-acb1-94058194b305 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:09.659521 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1350d-9a71-4b95-8751-9b2cd49c1095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:09.660185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1350d-9a71-4b95-8751-9b2cd49c1095 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:09.660462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1350d-9a71-4b95-8751-9b2cd49c1095 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:09.660738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1350d-9a71-4b95-8751-9b2cd49c1095 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:09.661539 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1350d-9a71-4b95-8751-9b2cd49c1095 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.domain.id=default, user.id=bb37d398d5fd456aa46fd2b5f5576815) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:09.661859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1350d-9a71-4b95-8751-9b2cd49c1095 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:09.663837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1350d-9a71-4b95-8751-9b2cd49c1095 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:09.673240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 28/57] 10.222.0.25 () {56 vars in 1322 bytes} [Sat Aug 6 01:04:09 2022] GET /identity/v3/role_assignments?user.id=bb37d398d5fd456aa46fd2b5f5576815&scope.domain.id=default&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:11.595088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4efb8-0a25-45d8-a0f9-aac62cfe5a74 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:11.595494 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4efb8-0a25-45d8-a0f9-aac62cfe5a74 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:11.595661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4efb8-0a25-45d8-a0f9-aac62cfe5a74 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:11.597070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 30/58] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:11.602306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbd614-3758-4b62-940d-fa9c329fe5cb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:11.602690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbd614-3758-4b62-940d-fa9c329fe5cb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:11.602869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbd614-3758-4b62-940d-fa9c329fe5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:11.615818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:11.621042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-46bbd614-3758-4b62-940d-fa9c329fe5cb None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:11.622211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-46bbd614-3758-4b62-940d-fa9c329fe5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:11.684980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 29/59] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:11.708712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4efb8-0a25-45d8-a0f9-aac62cfe5a74 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:11.805117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4efb8-0a25-45d8-a0f9-aac62cfe5a74 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:11.821328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:11.838349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f977baf-1560-4423-8581-a10e800127bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:11.838492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f977baf-1560-4423-8581-a10e800127bf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:11.838729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f977baf-1560-4423-8581-a10e800127bf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:11.839005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f977baf-1560-4423-8581-a10e800127bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:11.840171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f977baf-1560-4423-8581-a10e800127bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:11.841658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f977baf-1560-4423-8581-a10e800127bf None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:11.843215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 31/60] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:11.848914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbd614-3758-4b62-940d-fa9c329fe5cb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:11.942887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbd614-3758-4b62-940d-fa9c329fe5cb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:11.977081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f3b2d-aa89-4750-8a4a-07913c3063a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:11.977701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f3b2d-aa89-4750-8a4a-07913c3063a2 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:11.977991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f3b2d-aa89-4750-8a4a-07913c3063a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:11.978272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f3b2d-aa89-4750-8a4a-07913c3063a2 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:11.978973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f3b2d-aa89-4750-8a4a-07913c3063a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:11.980452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f3b2d-aa89-4750-8a4a-07913c3063a2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:11.993179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:12.132275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 30/61] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:04:11 2022] POST /identity/v3/users => generated 275 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:15.250118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0dc7-55bf-47f6-bf9b-3715b53ee227 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:15.250965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0dc7-55bf-47f6-bf9b-3715b53ee227 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:15.251416 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0dc7-55bf-47f6-bf9b-3715b53ee227 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:15.252965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 32/62] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:15.258995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7a310-0bc2-40d3-b80b-0727f48ca3e6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:15.259748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7a310-0bc2-40d3-b80b-0727f48ca3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:15.260307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7a310-0bc2-40d3-b80b-0727f48ca3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:15.288788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-28c7a310-0bc2-40d3-b80b-0727f48ca3e6 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:15.290329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-28c7a310-0bc2-40d3-b80b-0727f48ca3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:15.323801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff850-a391-4a1a-af6a-f29cbec07833 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:15.324402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff850-a391-4a1a-af6a-f29cbec07833 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:15.325144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff850-a391-4a1a-af6a-f29cbec07833 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:15.326736 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 33/63] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:15.343098 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d829da2-fc8a-4352-8c2c-4170eb73a421 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:15.343686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d829da2-fc8a-4352-8c2c-4170eb73a421 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:15.344194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d829da2-fc8a-4352-8c2c-4170eb73a421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:15.371041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0d829da2-fc8a-4352-8c2c-4170eb73a421 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:15.373287 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0d829da2-fc8a-4352-8c2c-4170eb73a421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:15.418226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 31/64] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:15.441020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7a310-0bc2-40d3-b80b-0727f48ca3e6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:15.470649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 34/65] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:15.496547 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d829da2-fc8a-4352-8c2c-4170eb73a421 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:15.571633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7a310-0bc2-40d3-b80b-0727f48ca3e6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:15.596037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:15.608367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aced54b-3b9b-4b9f-aeb7-9ff0835e1b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:15.609265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aced54b-3b9b-4b9f-aeb7-9ff0835e1b68 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:15.609705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aced54b-3b9b-4b9f-aeb7-9ff0835e1b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:15.610723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aced54b-3b9b-4b9f-aeb7-9ff0835e1b68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:15.622535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aced54b-3b9b-4b9f-aeb7-9ff0835e1b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0aced54b-3b9b-4b9f-aeb7-9ff0835e1b68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:15.628242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:15.629093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:15.629880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:15.633150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 32/66] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:15.635335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571bad5-6e01-43c0-ab43-2639c181fc06 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:15.635866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571bad5-6e01-43c0-ab43-2639c181fc06 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:15.636359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571bad5-6e01-43c0-ab43-2639c181fc06 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:15.638329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 33/67] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:15.639210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571bad5-6e01-43c0-ab43-2639c181fc06 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:15.641171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d829da2-fc8a-4352-8c2c-4170eb73a421 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:15.662209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:15.685273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25e10d-d69e-4b9a-b0cd-98dec9221eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:15.685950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25e10d-d69e-4b9a-b0cd-98dec9221eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:15.686243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25e10d-d69e-4b9a-b0cd-98dec9221eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:15.686529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25e10d-d69e-4b9a-b0cd-98dec9221eb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:15.700634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25e10d-d69e-4b9a-b0cd-98dec9221eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0f25e10d-d69e-4b9a-b0cd-98dec9221eb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:15.707232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:15.708283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 35/68] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:15.713884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d01379-6da3-4393-8835-15d0c2df5ad5 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:15.714575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d01379-6da3-4393-8835-15d0c2df5ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:15.714575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d01379-6da3-4393-8835-15d0c2df5ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:15.733590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-02d01379-6da3-4393-8835-15d0c2df5ad5 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:15.735280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-02d01379-6da3-4393-8835-15d0c2df5ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:15.767576 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571bad5-6e01-43c0-ab43-2639c181fc06 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:15.807156 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48270d24-3a04-4482-9b7c-7a9a0e760d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:15.808259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48270d24-3a04-4482-9b7c-7a9a0e760d33 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:15.808738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48270d24-3a04-4482-9b7c-7a9a0e760d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:15.809209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48270d24-3a04-4482-9b7c-7a9a0e760d33 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:15.810264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48270d24-3a04-4482-9b7c-7a9a0e760d33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:15.810902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48270d24-3a04-4482-9b7c-7a9a0e760d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:15.812550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48270d24-3a04-4482-9b7c-7a9a0e760d33 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:15.816129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 36/69] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:15.816824 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d01379-6da3-4393-8835-15d0c2df5ad5 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:15.823935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 34/70] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:15.834223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48270d24-3a04-4482-9b7c-7a9a0e760d33 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:15.926556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:15.934164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d01379-6da3-4393-8835-15d0c2df5ad5 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:15.942146 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:15.971259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123e844-e05a-4a5c-bc06-5916e2fa7025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:15.972176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e844-e05a-4a5c-bc06-5916e2fa7025 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:15.972661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e844-e05a-4a5c-bc06-5916e2fa7025 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:15.973074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e844-e05a-4a5c-bc06-5916e2fa7025 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:15.974015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123e844-e05a-4a5c-bc06-5916e2fa7025 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:15.974507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123e844-e05a-4a5c-bc06-5916e2fa7025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:15.976114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123e844-e05a-4a5c-bc06-5916e2fa7025 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:15.989723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 37/71] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:15.989723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123e844-e05a-4a5c-bc06-5916e2fa7025 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:15.993492 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48270d24-3a04-4482-9b7c-7a9a0e760d33 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:16.031701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823ff9b-8395-4d9c-a569-b7af34db99df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:16.032788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823ff9b-8395-4d9c-a569-b7af34db99df None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.033255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823ff9b-8395-4d9c-a569-b7af34db99df None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.033535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823ff9b-8395-4d9c-a569-b7af34db99df None admin] PATH_INFO: `/v3/users/a5948add9ba0485a99f1629cd6f9962b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.049206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823ff9b-8395-4d9c-a569-b7af34db99df None admin] RBAC: Authorizing `identity:get_user(user_id=a5948add9ba0485a99f1629cd6f9962b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:16.050663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823ff9b-8395-4d9c-a569-b7af34db99df None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:16.052465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 35/72] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:15 2022] GET /identity/v3/users/a5948add9ba0485a99f1629cd6f9962b => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:16.054180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bdfd0-517c-431b-8fb6-2b07d20a5350 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.054465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bdfd0-517c-431b-8fb6-2b07d20a5350 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.054742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bdfd0-517c-431b-8fb6-2b07d20a5350 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.056117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 36/73] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:16.060957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efc07a-9db6-4ec1-b48b-a46d8f43ed58 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.061249 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efc07a-9db6-4ec1-b48b-a46d8f43ed58 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.061528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efc07a-9db6-4ec1-b48b-a46d8f43ed58 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.062794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 37/74] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:16.063696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efc07a-9db6-4ec1-b48b-a46d8f43ed58 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:16.096421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123e844-e05a-4a5c-bc06-5916e2fa7025 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:16.134215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50840425-9e08-4da4-8fb7-3bd86645e6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:16.134956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50840425-9e08-4da4-8fb7-3bd86645e6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.135245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50840425-9e08-4da4-8fb7-3bd86645e6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.135518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50840425-9e08-4da4-8fb7-3bd86645e6f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.146373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50840425-9e08-4da4-8fb7-3bd86645e6f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-50840425-9e08-4da4-8fb7-3bd86645e6f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:16.151669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:16.152778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 38/75] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:16.153732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50840425-9e08-4da4-8fb7-3bd86645e6f1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:16.182692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efc07a-9db6-4ec1-b48b-a46d8f43ed58 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:16.183294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:16.184242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:16.216020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9ac81-40a8-4cf2-be2d-60c9a28ea754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:16.216753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9ac81-40a8-4cf2-be2d-60c9a28ea754 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.217042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9ac81-40a8-4cf2-be2d-60c9a28ea754 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.217316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9ac81-40a8-4cf2-be2d-60c9a28ea754 None admin] PATH_INFO: `/v3/users/bb37d398d5fd456aa46fd2b5f5576815` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.218490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9ac81-40a8-4cf2-be2d-60c9a28ea754 None admin] RBAC: Authorizing `identity:get_user(user_id=bb37d398d5fd456aa46fd2b5f5576815)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:16.219974 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9ac81-40a8-4cf2-be2d-60c9a28ea754 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:16.221584 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 38/76] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:16 2022] GET /identity/v3/users/bb37d398d5fd456aa46fd2b5f5576815 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:16.222684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64260325-c6c0-451c-8053-470d18820a99 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.222998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64260325-c6c0-451c-8053-470d18820a99 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.227405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64260325-c6c0-451c-8053-470d18820a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.242034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-64260325-c6c0-451c-8053-470d18820a99 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:16.243234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-64260325-c6c0-451c-8053-470d18820a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:16.261316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50840425-9e08-4da4-8fb7-3bd86645e6f1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:16.295171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d3d82-a30b-4e56-8c98-71194218ade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:16.295823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d3d82-a30b-4e56-8c98-71194218ade3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.296114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d3d82-a30b-4e56-8c98-71194218ade3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.296394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d3d82-a30b-4e56-8c98-71194218ade3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.307904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 39/77] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:16.313078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334c2ed-f0d0-46ea-9271-816e57071296 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.313367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334c2ed-f0d0-46ea-9271-816e57071296 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.313644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334c2ed-f0d0-46ea-9271-816e57071296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.331715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c334c2ed-f0d0-46ea-9271-816e57071296 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:16.332922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c334c2ed-f0d0-46ea-9271-816e57071296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:16.363890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d3d82-a30b-4e56-8c98-71194218ade3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:16.365350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d3d82-a30b-4e56-8c98-71194218ade3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:16.391676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:16.399678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 40/78] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:16.409403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334c2ed-f0d0-46ea-9271-816e57071296 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:16.433734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:16.463050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 39/79] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:16.464009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d3d82-a30b-4e56-8c98-71194218ade3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:16.502607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334c2ed-f0d0-46ea-9271-816e57071296 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:16.535224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac22585-00cc-412e-9343-3acbaae124b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:16.535869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac22585-00cc-412e-9343-3acbaae124b1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.536155 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac22585-00cc-412e-9343-3acbaae124b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.536430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac22585-00cc-412e-9343-3acbaae124b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.537169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac22585-00cc-412e-9343-3acbaae124b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:16.537474 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac22585-00cc-412e-9343-3acbaae124b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:16.538837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac22585-00cc-412e-9343-3acbaae124b1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:16.546611 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 41/80] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:16.547378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac22585-00cc-412e-9343-3acbaae124b1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:16.563568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d3d82-a30b-4e56-8c98-71194218ade3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:16.592063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:16.599438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d7420-3048-4573-ae5e-6871855b042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:16.600106 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7420-3048-4573-ae5e-6871855b042a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.603861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7420-3048-4573-ae5e-6871855b042a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.603861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7420-3048-4573-ae5e-6871855b042a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.637694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:16.645823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac22585-00cc-412e-9343-3acbaae124b1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:16.666893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d7420-3048-4573-ae5e-6871855b042a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:16.668409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d7420-3048-4573-ae5e-6871855b042a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:16.678563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f58ce6-190d-46a6-be78-c5a200ebdb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:16.679200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f58ce6-190d-46a6-be78-c5a200ebdb11 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.679490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f58ce6-190d-46a6-be78-c5a200ebdb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.679764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f58ce6-190d-46a6-be78-c5a200ebdb11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.689444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f58ce6-190d-46a6-be78-c5a200ebdb11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-84f58ce6-190d-46a6-be78-c5a200ebdb11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:16.694862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:16.695710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:16.696728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 42/81] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:16.701796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f58ce6-190d-46a6-be78-c5a200ebdb11 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:16.765551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 40/82] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:16.766446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d7420-3048-4573-ae5e-6871855b042a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:16.791991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:16.802463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f58ce6-190d-46a6-be78-c5a200ebdb11 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:16.836152 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001d648-9315-4f92-9379-688adfee5580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:16.836783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001d648-9315-4f92-9379-688adfee5580 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.837066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001d648-9315-4f92-9379-688adfee5580 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.837341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001d648-9315-4f92-9379-688adfee5580 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.838496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001d648-9315-4f92-9379-688adfee5580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:16.839982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001d648-9315-4f92-9379-688adfee5580 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:16.841565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 43/83] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:16.846110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001d648-9315-4f92-9379-688adfee5580 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:16.863086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d7420-3048-4573-ae5e-6871855b042a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:16.866294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:16.896259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864805ad-90eb-47e9-8573-ebe0185dbb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:16.896859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864805ad-90eb-47e9-8573-ebe0185dbb49 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.897148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864805ad-90eb-47e9-8573-ebe0185dbb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.897428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864805ad-90eb-47e9-8573-ebe0185dbb49 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.898596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864805ad-90eb-47e9-8573-ebe0185dbb49 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:16.900099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864805ad-90eb-47e9-8573-ebe0185dbb49 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:16.901757 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 41/84] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:16 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:16.902570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864805ad-90eb-47e9-8573-ebe0185dbb49 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:16.942005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001d648-9315-4f92-9379-688adfee5580 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:16.975639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa853a1-3941-42c3-a775-60f1538e6514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:16.976298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa853a1-3941-42c3-a775-60f1538e6514 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:16.976587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa853a1-3941-42c3-a775-60f1538e6514 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:16.976861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa853a1-3941-42c3-a775-60f1538e6514 None admin] PATH_INFO: `/v3/users/a5948add9ba0485a99f1629cd6f9962b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:16.978050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa853a1-3941-42c3-a775-60f1538e6514 None admin] RBAC: Authorizing `identity:get_user(user_id=a5948add9ba0485a99f1629cd6f9962b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:16.979493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa853a1-3941-42c3-a775-60f1538e6514 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:16.981678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 44/85] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:16 2022] GET /identity/v3/users/a5948add9ba0485a99f1629cd6f9962b => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:16.981881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa853a1-3941-42c3-a775-60f1538e6514 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:16.999766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:17.005302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864805ad-90eb-47e9-8573-ebe0185dbb49 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:17.038224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f3e5e-a475-411e-b619-048c8d51f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:17.038926 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f3e5e-a475-411e-b619-048c8d51f380 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:17.039219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f3e5e-a475-411e-b619-048c8d51f380 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:17.039490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f3e5e-a475-411e-b619-048c8d51f380 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:17.040230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f3e5e-a475-411e-b619-048c8d51f380 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:17.040543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f3e5e-a475-411e-b619-048c8d51f380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:17.041923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f3e5e-a475-411e-b619-048c8d51f380 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:17.049837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 42/86] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:17.049837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f3e5e-a475-411e-b619-048c8d51f380 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:17.068885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:17.080731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa853a1-3941-42c3-a775-60f1538e6514 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:17.114837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059a822-50a1-4f8e-8f3c-b436529aff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:17.116063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059a822-50a1-4f8e-8f3c-b436529aff3d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:17.116348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059a822-50a1-4f8e-8f3c-b436529aff3d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:17.116632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059a822-50a1-4f8e-8f3c-b436529aff3d None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:17.117799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059a822-50a1-4f8e-8f3c-b436529aff3d None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:17.119247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059a822-50a1-4f8e-8f3c-b436529aff3d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:17.120940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 45/87] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:16 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:17.125731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059a822-50a1-4f8e-8f3c-b436529aff3d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:17.153915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f3e5e-a475-411e-b619-048c8d51f380 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:17.187112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3d49c-f943-4890-8cf0-ef546f897245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:17.187906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3d49c-f943-4890-8cf0-ef546f897245 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:17.188194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3d49c-f943-4890-8cf0-ef546f897245 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:17.188472 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3d49c-f943-4890-8cf0-ef546f897245 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:17.189160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3d49c-f943-4890-8cf0-ef546f897245 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:17.190616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3d49c-f943-4890-8cf0-ef546f897245 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:17.219355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059a822-50a1-4f8e-8f3c-b436529aff3d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:17.252127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e45e30-97ee-4934-9d6f-b98ac81672a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:17.252744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e45e30-97ee-4934-9d6f-b98ac81672a2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:17.253031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e45e30-97ee-4934-9d6f-b98ac81672a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:17.253315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e45e30-97ee-4934-9d6f-b98ac81672a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:17.254107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e45e30-97ee-4934-9d6f-b98ac81672a2 None admin] RBAC: Adding query filter params (role.id=b5b241afd53c456e9a9fa3033eea74ad, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b, user.id=a5948add9ba0485a99f1629cd6f9962b) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:17.254427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e45e30-97ee-4934-9d6f-b98ac81672a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:17.255877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e45e30-97ee-4934-9d6f-b98ac81672a2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:17.261080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 46/88] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:17 2022] GET /identity/v3/role_assignments?user.id=a5948add9ba0485a99f1629cd6f9962b&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=b5b241afd53c456e9a9fa3033eea74ad => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:17.261852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e45e30-97ee-4934-9d6f-b98ac81672a2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:17.281571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 43/89] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:04:17 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:17.282279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3d49c-f943-4890-8cf0-ef546f897245 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:17.299623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:17.310149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:17.398270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e45e30-97ee-4934-9d6f-b98ac81672a2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:17.447513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3d49c-f943-4890-8cf0-ef546f897245 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:17.461514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0645e7d-cfdb-4fd1-ac74-52e77c322d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:17.463032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0645e7d-cfdb-4fd1-ac74-52e77c322d41 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:17.463606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0645e7d-cfdb-4fd1-ac74-52e77c322d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:17.464208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0645e7d-cfdb-4fd1-ac74-52e77c322d41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:17.491300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c784d-f998-4532-936d-a170057a8d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:17.492966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c784d-f998-4532-936d-a170057a8d20 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:17.492966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c784d-f998-4532-936d-a170057a8d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:17.492966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c784d-f998-4532-936d-a170057a8d20 None admin] PATH_INFO: `/v3/users/a5948add9ba0485a99f1629cd6f9962b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:17.494356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c784d-f998-4532-936d-a170057a8d20 None admin] RBAC: Authorizing `identity:get_user(user_id=a5948add9ba0485a99f1629cd6f9962b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:17.495534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c784d-f998-4532-936d-a170057a8d20 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:17.497358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 44/90] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:17 2022] GET /identity/v3/users/a5948add9ba0485a99f1629cd6f9962b => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:17.498626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c784d-f998-4532-936d-a170057a8d20 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:17.557368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0645e7d-cfdb-4fd1-ac74-52e77c322d41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:17.559135 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0645e7d-cfdb-4fd1-ac74-52e77c322d41 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:17.598562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:17.601112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:17.626736 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c784d-f998-4532-936d-a170057a8d20 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:17.669886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb73a0-a29c-4a68-9891-3237ea17bd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:17.670506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb73a0-a29c-4a68-9891-3237ea17bd5d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:17.670797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb73a0-a29c-4a68-9891-3237ea17bd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:17.671077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb73a0-a29c-4a68-9891-3237ea17bd5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:17.671922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb73a0-a29c-4a68-9891-3237ea17bd5d None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b, user.id=bb37d398d5fd456aa46fd2b5f5576815) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:17.672242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb73a0-a29c-4a68-9891-3237ea17bd5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:17.673815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb73a0-a29c-4a68-9891-3237ea17bd5d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:17.681004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 45/91] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:17 2022] GET /identity/v3/role_assignments?user.id=bb37d398d5fd456aa46fd2b5f5576815&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:17.681004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb73a0-a29c-4a68-9891-3237ea17bd5d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:17.706198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 47/92] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:17.712799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0645e7d-cfdb-4fd1-ac74-52e77c322d41 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:17.804480 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb73a0-a29c-4a68-9891-3237ea17bd5d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:17.892605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1415e44-136d-402b-afda-bc0499fd370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:17.893253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1415e44-136d-402b-afda-bc0499fd370c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:17.893538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1415e44-136d-402b-afda-bc0499fd370c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:17.893814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1415e44-136d-402b-afda-bc0499fd370c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:17.927715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0645e7d-cfdb-4fd1-ac74-52e77c322d41 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:17.956998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:17.971311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:18.001827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71d2a5-bfb4-41aa-8537-1d3db7911f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:18.002501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71d2a5-bfb4-41aa-8537-1d3db7911f86 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:18.002789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71d2a5-bfb4-41aa-8537-1d3db7911f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:18.003074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71d2a5-bfb4-41aa-8537-1d3db7911f86 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:18.008544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71d2a5-bfb4-41aa-8537-1d3db7911f86 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:18.010535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71d2a5-bfb4-41aa-8537-1d3db7911f86 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:18.012320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 48/93] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:17 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:18.024427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71d2a5-bfb4-41aa-8537-1d3db7911f86 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:18.076666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1415e44-136d-402b-afda-bc0499fd370c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:18.078190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1415e44-136d-402b-afda-bc0499fd370c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:18.208058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71d2a5-bfb4-41aa-8537-1d3db7911f86 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:18.261529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034050c-9083-43b3-ae2d-1ac224c0d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:18.262158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034050c-9083-43b3-ae2d-1ac224c0d801 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:18.262446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034050c-9083-43b3-ae2d-1ac224c0d801 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:18.262730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034050c-9083-43b3-ae2d-1ac224c0d801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:18.263529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034050c-9083-43b3-ae2d-1ac224c0d801 None admin] RBAC: Adding query filter params (role.id=512a743b8ca748f8b793681748be7906, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b, user.id=a5948add9ba0485a99f1629cd6f9962b) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:18.264206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034050c-9083-43b3-ae2d-1ac224c0d801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:18.266436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034050c-9083-43b3-ae2d-1ac224c0d801 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:18.277937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 49/94] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:18 2022] GET /identity/v3/role_assignments?user.id=a5948add9ba0485a99f1629cd6f9962b&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=512a743b8ca748f8b793681748be7906 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:18.286893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 46/95] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:18.292330 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034050c-9083-43b3-ae2d-1ac224c0d801 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:18.365781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:18.428882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034050c-9083-43b3-ae2d-1ac224c0d801 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:18.493461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9d9b6-45c0-4ab4-b71d-65a8ef4df5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:18.494040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9d9b6-45c0-4ab4-b71d-65a8ef4df5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:18.494332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9d9b6-45c0-4ab4-b71d-65a8ef4df5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:18.494617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9d9b6-45c0-4ab4-b71d-65a8ef4df5d2 None admin] PATH_INFO: `/v3/projects/cefb9581523149d38c64301b2796ae2f` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:18.500601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe9d9b6-45c0-4ab4-b71d-65a8ef4df5d2 None admin] RBAC: Authorizing `identity:get_project(project_id=cefb9581523149d38c64301b2796ae2f)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:18.502116 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe9d9b6-45c0-4ab4-b71d-65a8ef4df5d2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:18.504261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 50/96] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:18 2022] GET /identity/v3/projects/cefb9581523149d38c64301b2796ae2f => generated 312 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:18.510192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1415e44-136d-402b-afda-bc0499fd370c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:18.575355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:18.651834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1415e44-136d-402b-afda-bc0499fd370c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:18.691243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32856d-754a-4c0e-8977-245ef84ea8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:18.691901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32856d-754a-4c0e-8977-245ef84ea8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:18.692189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32856d-754a-4c0e-8977-245ef84ea8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:18.692465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32856d-754a-4c0e-8977-245ef84ea8b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:18.693292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32856d-754a-4c0e-8977-245ef84ea8b2 None admin] RBAC: Adding query filter params (role.id=512a743b8ca748f8b793681748be7906, scope.project.id=cefb9581523149d38c64301b2796ae2f, user.id=a5948add9ba0485a99f1629cd6f9962b) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:18.693606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32856d-754a-4c0e-8977-245ef84ea8b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:18.695112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32856d-754a-4c0e-8977-245ef84ea8b2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:18.700881 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 47/97] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:18 2022] GET /identity/v3/role_assignments?user.id=a5948add9ba0485a99f1629cd6f9962b&scope.project.id=cefb9581523149d38c64301b2796ae2f&role.id=512a743b8ca748f8b793681748be7906 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:20.120578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba91af-a044-4a5b-9b0e-1e5af207b26b None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.121626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba91af-a044-4a5b-9b0e-1e5af207b26b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.122136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba91af-a044-4a5b-9b0e-1e5af207b26b None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.123719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 51/98] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:20.137242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911a6b7-fd08-4413-aaca-311f66b50781 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.138136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911a6b7-fd08-4413-aaca-311f66b50781 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.138655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911a6b7-fd08-4413-aaca-311f66b50781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.161694 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a911a6b7-fd08-4413-aaca-311f66b50781 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:20.163752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a911a6b7-fd08-4413-aaca-311f66b50781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:20.240526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 48/99] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:20.282595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba91af-a044-4a5b-9b0e-1e5af207b26b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:20.291661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c11c1-a686-4030-8a9f-4c6138ef72fd None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.291996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c11c1-a686-4030-8a9f-4c6138ef72fd None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.292265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c11c1-a686-4030-8a9f-4c6138ef72fd None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.293631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 49/100] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:20.298839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f16be-3bcf-4211-9503-99ee23df6349 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.299123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f16be-3bcf-4211-9503-99ee23df6349 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.299403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f16be-3bcf-4211-9503-99ee23df6349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.323995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2f8f16be-3bcf-4211-9503-99ee23df6349 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:20.325471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2f8f16be-3bcf-4211-9503-99ee23df6349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:20.385160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:20.442846 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 50/101] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:20.445039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b078e-f963-4169-85b9-641b5b57604a None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.445343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b078e-f963-4169-85b9-641b5b57604a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.445626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b078e-f963-4169-85b9-641b5b57604a None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.447007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 51/102] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:20.452074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3065f-daf7-4467-a468-ac1773adea08 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.452362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3065f-daf7-4467-a468-ac1773adea08 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.452644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3065f-daf7-4467-a468-ac1773adea08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.486405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1aa3065f-daf7-4467-a468-ac1773adea08 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:20.487582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1aa3065f-daf7-4467-a468-ac1773adea08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:20.499748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba91af-a044-4a5b-9b0e-1e5af207b26b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:20.528527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:20.542898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776f9ea-8a8f-4c57-b4e8-9eb5f4e16a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:20.543609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776f9ea-8a8f-4c57-b4e8-9eb5f4e16a81 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.543996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776f9ea-8a8f-4c57-b4e8-9eb5f4e16a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.544273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776f9ea-8a8f-4c57-b4e8-9eb5f4e16a81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.554944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776f9ea-8a8f-4c57-b4e8-9eb5f4e16a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-7776f9ea-8a8f-4c57-b4e8-9eb5f4e16a81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:20.560082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:20.561125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:20.562017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:20.562608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 52/103] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:20.563313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776f9ea-8a8f-4c57-b4e8-9eb5f4e16a81 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:20.590660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 52/104] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:20.591167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56279030-8f85-459b-becb-94cd85654796 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.591648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56279030-8f85-459b-becb-94cd85654796 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.592111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56279030-8f85-459b-becb-94cd85654796 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.593562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 53/105] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:20.594434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56279030-8f85-459b-becb-94cd85654796 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:20.690237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776f9ea-8a8f-4c57-b4e8-9eb5f4e16a81 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:20.727929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:20.728484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56279030-8f85-459b-becb-94cd85654796 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:20.731866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b85e0-9066-4234-96f3-8bc39afbe48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:20.732767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b85e0-9066-4234-96f3-8bc39afbe48f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.733247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b85e0-9066-4234-96f3-8bc39afbe48f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.733716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b85e0-9066-4234-96f3-8bc39afbe48f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.744545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b85e0-9066-4234-96f3-8bc39afbe48f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-194b85e0-9066-4234-96f3-8bc39afbe48f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:20.751304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:20.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:20.753013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:20.755417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 53/106] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:20.757602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ea933-8529-4dd9-931f-471ca4f9f9d3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.758093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ea933-8529-4dd9-931f-471ca4f9f9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.758558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ea933-8529-4dd9-931f-471ca4f9f9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.768654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30b897-5ddc-47b9-b816-27b7e8cd1acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:20.769342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30b897-5ddc-47b9-b816-27b7e8cd1acb None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:20.769630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30b897-5ddc-47b9-b816-27b7e8cd1acb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:20.769908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30b897-5ddc-47b9-b816-27b7e8cd1acb None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:20.770642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30b897-5ddc-47b9-b816-27b7e8cd1acb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:20.770954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30b897-5ddc-47b9-b816-27b7e8cd1acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:20.773204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30b897-5ddc-47b9-b816-27b7e8cd1acb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:20.781410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c57ea933-8529-4dd9-931f-471ca4f9f9d3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:20.782656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 54/107] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:20.783381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c57ea933-8529-4dd9-931f-471ca4f9f9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:20.787903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30b897-5ddc-47b9-b816-27b7e8cd1acb None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:20.821840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:20.858531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 54/108] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:20.859949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ea933-8529-4dd9-931f-471ca4f9f9d3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.013493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ea933-8529-4dd9-931f-471ca4f9f9d3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.017353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30b897-5ddc-47b9-b816-27b7e8cd1acb None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.056242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294f78d-5116-4203-a79e-6a6105c2b346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.056876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294f78d-5116-4203-a79e-6a6105c2b346 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.057158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294f78d-5116-4203-a79e-6a6105c2b346 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.057429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294f78d-5116-4203-a79e-6a6105c2b346 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.058154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294f78d-5116-4203-a79e-6a6105c2b346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:21.058463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294f78d-5116-4203-a79e-6a6105c2b346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:21.059930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294f78d-5116-4203-a79e-6a6105c2b346 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:21.061274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bc5a1-2f25-426c-a47a-ee93e50e722b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.062534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bc5a1-2f25-426c-a47a-ee93e50e722b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.063026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bc5a1-2f25-426c-a47a-ee93e50e722b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.063497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bc5a1-2f25-426c-a47a-ee93e50e722b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.069625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 55/109] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:21.070346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294f78d-5116-4203-a79e-6a6105c2b346 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.078386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bc5a1-2f25-426c-a47a-ee93e50e722b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-dd9bc5a1-2f25-426c-a47a-ee93e50e722b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:21.085651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:21.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:21.087267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:21.089234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 55/110] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:21.089874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bc5a1-2f25-426c-a47a-ee93e50e722b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.111254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:21.119678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:21.186597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294f78d-5116-4203-a79e-6a6105c2b346 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.197849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bc5a1-2f25-426c-a47a-ee93e50e722b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.225730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083172ce-dbaf-491f-90fe-a09e2b7d1590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.226729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083172ce-dbaf-491f-90fe-a09e2b7d1590 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.227207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083172ce-dbaf-491f-90fe-a09e2b7d1590 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.227755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083172ce-dbaf-491f-90fe-a09e2b7d1590 None admin] PATH_INFO: `/v3/users/a5948add9ba0485a99f1629cd6f9962b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.229288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083172ce-dbaf-491f-90fe-a09e2b7d1590 None admin] RBAC: Authorizing `identity:get_user(user_id=a5948add9ba0485a99f1629cd6f9962b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:21.231333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083172ce-dbaf-491f-90fe-a09e2b7d1590 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:21.233330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174015-0d18-4644-a3ea-5d6c0638b629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.233600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 56/111] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:21 2022] GET /identity/v3/users/a5948add9ba0485a99f1629cd6f9962b => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:21.233985 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174015-0d18-4644-a3ea-5d6c0638b629 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.234272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174015-0d18-4644-a3ea-5d6c0638b629 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.234550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174015-0d18-4644-a3ea-5d6c0638b629 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.235261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083172ce-dbaf-491f-90fe-a09e2b7d1590 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.241030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c174015-0d18-4644-a3ea-5d6c0638b629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:21.242523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c174015-0d18-4644-a3ea-5d6c0638b629 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:21.248890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 56/112] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:21.249742 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174015-0d18-4644-a3ea-5d6c0638b629 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.338423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:21.341412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083172ce-dbaf-491f-90fe-a09e2b7d1590 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.346289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:21.356401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174015-0d18-4644-a3ea-5d6c0638b629 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.375717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b26e3-e042-4eab-bf26-ac589cb13de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.376393 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b26e3-e042-4eab-bf26-ac589cb13de1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.376677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b26e3-e042-4eab-bf26-ac589cb13de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.376952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b26e3-e042-4eab-bf26-ac589cb13de1 None admin] PATH_INFO: `/v3/users/bb37d398d5fd456aa46fd2b5f5576815` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.378102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b26e3-e042-4eab-bf26-ac589cb13de1 None admin] RBAC: Authorizing `identity:get_user(user_id=bb37d398d5fd456aa46fd2b5f5576815)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:21.379557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b26e3-e042-4eab-bf26-ac589cb13de1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:21.381207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 57/113] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:21 2022] GET /identity/v3/users/bb37d398d5fd456aa46fd2b5f5576815 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:21.381964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b26e3-e042-4eab-bf26-ac589cb13de1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.397887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326c689-f5d4-48a6-ad83-b8754db79bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.398797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326c689-f5d4-48a6-ad83-b8754db79bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.399238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326c689-f5d4-48a6-ad83-b8754db79bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.399523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326c689-f5d4-48a6-ad83-b8754db79bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.400346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326c689-f5d4-48a6-ad83-b8754db79bc1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:21.400801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326c689-f5d4-48a6-ad83-b8754db79bc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:21.402317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326c689-f5d4-48a6-ad83-b8754db79bc1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:21.411082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 57/114] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:21.411082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326c689-f5d4-48a6-ad83-b8754db79bc1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.492045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b26e3-e042-4eab-bf26-ac589cb13de1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.517984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326c689-f5d4-48a6-ad83-b8754db79bc1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.529932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8f13d-1269-4fac-b199-b6d98138d406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.530584 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8f13d-1269-4fac-b199-b6d98138d406 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.530864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8f13d-1269-4fac-b199-b6d98138d406 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.531141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8f13d-1269-4fac-b199-b6d98138d406 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.532329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8f13d-1269-4fac-b199-b6d98138d406 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:21.533759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8f13d-1269-4fac-b199-b6d98138d406 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:21.535413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 58/115] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:21 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:21.536645 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f268c88-a318-42f6-a497-a11c09eaa173 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.537444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f268c88-a318-42f6-a497-a11c09eaa173 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.537730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f268c88-a318-42f6-a497-a11c09eaa173 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.542932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 59/116] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:21.542932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f268c88-a318-42f6-a497-a11c09eaa173 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.561482 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c533-58d5-4a20-871e-1604579d047e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.562122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c533-58d5-4a20-871e-1604579d047e None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.562410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c533-58d5-4a20-871e-1604579d047e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.562689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c533-58d5-4a20-871e-1604579d047e None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.563376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9c533-58d5-4a20-871e-1604579d047e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:21.564949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9c533-58d5-4a20-871e-1604579d047e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:21.587928 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:21.655246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 58/117] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:04:21 2022] POST /identity/v3/projects => generated 302 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:21.655462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f268c88-a318-42f6-a497-a11c09eaa173 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.655556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c533-58d5-4a20-871e-1604579d047e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.659477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:21.687326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286a1c0-3b31-47c8-84f5-7eaa82252656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.688136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286a1c0-3b31-47c8-84f5-7eaa82252656 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.688426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286a1c0-3b31-47c8-84f5-7eaa82252656 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.688777 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286a1c0-3b31-47c8-84f5-7eaa82252656 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.690112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286a1c0-3b31-47c8-84f5-7eaa82252656 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:21.691620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286a1c0-3b31-47c8-84f5-7eaa82252656 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:21.698275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 60/118] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:21 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:21.698275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286a1c0-3b31-47c8-84f5-7eaa82252656 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.776831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c533-58d5-4a20-871e-1604579d047e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.814510 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286a1c0-3b31-47c8-84f5-7eaa82252656 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:21.816336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5052977-18f8-4304-b4e0-7a8a729929de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.816994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5052977-18f8-4304-b4e0-7a8a729929de None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.817280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5052977-18f8-4304-b4e0-7a8a729929de None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.817559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5052977-18f8-4304-b4e0-7a8a729929de None admin] PATH_INFO: `/v3/users/a05cd95cd10645a392f5bcfee57786e7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.818618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:21.836740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5052977-18f8-4304-b4e0-7a8a729929de None admin] RBAC: Authorizing `identity:get_user(user_id=a05cd95cd10645a392f5bcfee57786e7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:21.838252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5052977-18f8-4304-b4e0-7a8a729929de None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:21.840066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 59/119] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:21 2022] GET /identity/v3/users/a05cd95cd10645a392f5bcfee57786e7 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:21.841980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3296bfb-e79a-482c-8729-a65fbb053b73 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.841980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3296bfb-e79a-482c-8729-a65fbb053b73 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.841980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3296bfb-e79a-482c-8729-a65fbb053b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.860447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d3296bfb-e79a-482c-8729-a65fbb053b73 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:21.861431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc23a9-4125-4f33-838a-78da79f5b56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:21.861656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d3296bfb-e79a-482c-8729-a65fbb053b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:21.862121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc23a9-4125-4f33-838a-78da79f5b56e None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.862410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc23a9-4125-4f33-838a-78da79f5b56e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.862692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc23a9-4125-4f33-838a-78da79f5b56e None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/users/a5948add9ba0485a99f1629cd6f9962b/roles/b5b241afd53c456e9a9fa3033eea74ad` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.865747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc23a9-4125-4f33-838a-78da79f5b56e None admin] RBAC: Authorizing `identity:create_grant(project_id=7fa1f9bede2243f0ba30c5f573dc695b, user_id=a5948add9ba0485a99f1629cd6f9962b, role_id=b5b241afd53c456e9a9fa3033eea74ad)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:21.867193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc23a9-4125-4f33-838a-78da79f5b56e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:21.921337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:21.931339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 60/120] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:21.935806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2714b-9156-4327-9e38-0ac9418bc5d8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:21.935806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2714b-9156-4327-9e38-0ac9418bc5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:21.935806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2714b-9156-4327-9e38-0ac9418bc5d8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:21.935806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 61/121] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:21.939445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 61/122] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:04:21 2022] PUT /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/users/a5948add9ba0485a99f1629cd6f9962b/roles/b5b241afd53c456e9a9fa3033eea74ad => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:21.939631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2714b-9156-4327-9e38-0ac9418bc5d8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:21.940697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc23a9-4125-4f33-838a-78da79f5b56e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:22.074889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc23a9-4125-4f33-838a-78da79f5b56e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:22.080890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2714b-9156-4327-9e38-0ac9418bc5d8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:22.120644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728a6bc-d666-4f7f-ad99-c1e7510e5ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:22.121620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a6bc-d666-4f7f-ad99-c1e7510e5ceb None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:22.121910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a6bc-d666-4f7f-ad99-c1e7510e5ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:22.122018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae3751-fa77-4d8d-9d17-23a5539a3e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:22.122189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a6bc-d666-4f7f-ad99-c1e7510e5ceb None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/users/bb37d398d5fd456aa46fd2b5f5576815/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:22.122580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae3751-fa77-4d8d-9d17-23a5539a3e0e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:22.122867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae3751-fa77-4d8d-9d17-23a5539a3e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:22.123145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae3751-fa77-4d8d-9d17-23a5539a3e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:22.124416 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728a6bc-d666-4f7f-ad99-c1e7510e5ceb None admin] RBAC: Authorizing `identity:create_grant(project_id=7fa1f9bede2243f0ba30c5f573dc695b, user_id=bb37d398d5fd456aa46fd2b5f5576815, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:22.126079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728a6bc-d666-4f7f-ad99-c1e7510e5ceb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:22.163939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:22.201795 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae3751-fa77-4d8d-9d17-23a5539a3e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:22.203257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae3751-fa77-4d8d-9d17-23a5539a3e0e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:22.225123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 62/123] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:04:21 2022] PUT /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/users/bb37d398d5fd456aa46fd2b5f5576815/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:22.227705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48325-7a54-4eb2-bab1-e5a3fc697b01 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:22.228631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48325-7a54-4eb2-bab1-e5a3fc697b01 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:22.229379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48325-7a54-4eb2-bab1-e5a3fc697b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:22.258263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c9d48325-7a54-4eb2-bab1-e5a3fc697b01 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:22.260684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c9d48325-7a54-4eb2-bab1-e5a3fc697b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:22.367655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:22.376687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 62/124] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:22.377649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae3751-fa77-4d8d-9d17-23a5539a3e0e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:22.413905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 63/125] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:22.415243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48325-7a54-4eb2-bab1-e5a3fc697b01 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:22.446748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:22.490780 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae3751-fa77-4d8d-9d17-23a5539a3e0e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:22.533360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4ca94-b2ea-4174-9850-2397a2de01a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:22.534307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4ca94-b2ea-4174-9850-2397a2de01a4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:22.534819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4ca94-b2ea-4174-9850-2397a2de01a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:22.535296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4ca94-b2ea-4174-9850-2397a2de01a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:22.546965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4ca94-b2ea-4174-9850-2397a2de01a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:22.553158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48325-7a54-4eb2-bab1-e5a3fc697b01 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-30d4ca94-b2ea-4174-9850-2397a2de01a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:22.553586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:22.554426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:22.555204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:22.557616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 63/126] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:22.558769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4ca94-b2ea-4174-9850-2397a2de01a4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:22.590002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159a9dd-25de-4d7f-9160-665babe413a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:22.590763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159a9dd-25de-4d7f-9160-665babe413a8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:22.591047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159a9dd-25de-4d7f-9160-665babe413a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:22.591324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159a9dd-25de-4d7f-9160-665babe413a8 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:22.592765 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159a9dd-25de-4d7f-9160-665babe413a8 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:22.594150 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159a9dd-25de-4d7f-9160-665babe413a8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:22.595834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 64/127] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:22 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:22.596611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159a9dd-25de-4d7f-9160-665babe413a8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:22.651195 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:22.670465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4ca94-b2ea-4174-9850-2397a2de01a4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:22.696137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:22.703554 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159a9dd-25de-4d7f-9160-665babe413a8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:22.708491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7113f03-449f-4bc0-a1e0-8ccb58a95835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:22.709444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7113f03-449f-4bc0-a1e0-8ccb58a95835 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:22.709893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7113f03-449f-4bc0-a1e0-8ccb58a95835 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:22.710334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7113f03-449f-4bc0-a1e0-8ccb58a95835 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:22.721276 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7113f03-449f-4bc0-a1e0-8ccb58a95835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f7113f03-449f-4bc0-a1e0-8ccb58a95835 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:22.727272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:22.728180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:22.728973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:22.732414 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 64/128] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:22.733581 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7113f03-449f-4bc0-a1e0-8ccb58a95835 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:22.744879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848140ac-d483-49a0-a93f-59e6e4e3b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:22.745646 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848140ac-d483-49a0-a93f-59e6e4e3b94c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:22.745754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848140ac-d483-49a0-a93f-59e6e4e3b94c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:22.745976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848140ac-d483-49a0-a93f-59e6e4e3b94c None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:22.746883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848140ac-d483-49a0-a93f-59e6e4e3b94c None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:22.747198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848140ac-d483-49a0-a93f-59e6e4e3b94c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:22.748823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848140ac-d483-49a0-a93f-59e6e4e3b94c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:22.758475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 65/129] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:22.759295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848140ac-d483-49a0-a93f-59e6e4e3b94c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:22.852196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7113f03-449f-4bc0-a1e0-8ccb58a95835 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:22.876528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848140ac-d483-49a0-a93f-59e6e4e3b94c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:22.890492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:22.891014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac772e-054b-4ae0-9bbe-7f3292901a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:22.891894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac772e-054b-4ae0-9bbe-7f3292901a27 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:22.892407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac772e-054b-4ae0-9bbe-7f3292901a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:22.892887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac772e-054b-4ae0-9bbe-7f3292901a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:22.893885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac772e-054b-4ae0-9bbe-7f3292901a27 None admin] RBAC: Adding query filter params (role.id=324a803dd87d4027b73d871fe46ea228, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b, user.id=a05cd95cd10645a392f5bcfee57786e7) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:22.894401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac772e-054b-4ae0-9bbe-7f3292901a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:22.896117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac772e-054b-4ae0-9bbe-7f3292901a27 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:22.902702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 65/130] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:22 2022] GET /identity/v3/role_assignments?user.id=a05cd95cd10645a392f5bcfee57786e7&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=324a803dd87d4027b73d871fe46ea228 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:22.903944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac772e-054b-4ae0-9bbe-7f3292901a27 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:22.913844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ee8e0-47ef-4487-bd2b-ecb7edb6b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:22.915000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ee8e0-47ef-4487-bd2b-ecb7edb6b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:22.915554 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ee8e0-47ef-4487-bd2b-ecb7edb6b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:22.916076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ee8e0-47ef-4487-bd2b-ecb7edb6b1a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:22.917037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ee8e0-47ef-4487-bd2b-ecb7edb6b1a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:22.917580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ee8e0-47ef-4487-bd2b-ecb7edb6b1a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:22.919258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ee8e0-47ef-4487-bd2b-ecb7edb6b1a7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:22.931202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 66/131] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:22.936510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ee8e0-47ef-4487-bd2b-ecb7edb6b1a7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:23.018638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:23.076560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac772e-054b-4ae0-9bbe-7f3292901a27 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:23.117586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e50e70-a41b-4240-b599-ae0ddd875c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:23.118606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e50e70-a41b-4240-b599-ae0ddd875c5b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:23.119117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e50e70-a41b-4240-b599-ae0ddd875c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:23.119619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e50e70-a41b-4240-b599-ae0ddd875c5b None admin] PATH_INFO: `/v3/users/a5948add9ba0485a99f1629cd6f9962b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:23.122054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e50e70-a41b-4240-b599-ae0ddd875c5b None admin] RBAC: Authorizing `identity:get_user(user_id=a5948add9ba0485a99f1629cd6f9962b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:23.123922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e50e70-a41b-4240-b599-ae0ddd875c5b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:23.125955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 66/132] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:22 2022] GET /identity/v3/users/a5948add9ba0485a99f1629cd6f9962b => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:23.131958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e50e70-a41b-4240-b599-ae0ddd875c5b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:23.150055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ee8e0-47ef-4487-bd2b-ecb7edb6b1a7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:23.208436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43641d59-8972-4bc9-a116-85fa02537adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:23.209624 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43641d59-8972-4bc9-a116-85fa02537adc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:23.209909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43641d59-8972-4bc9-a116-85fa02537adc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:23.210185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43641d59-8972-4bc9-a116-85fa02537adc None admin] PATH_INFO: `/v3/users/a5948add9ba0485a99f1629cd6f9962b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:23.211509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43641d59-8972-4bc9-a116-85fa02537adc None admin] RBAC: Authorizing `identity:get_user(user_id=a5948add9ba0485a99f1629cd6f9962b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:23.213001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43641d59-8972-4bc9-a116-85fa02537adc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:23.215347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 67/133] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:22 2022] GET /identity/v3/users/a5948add9ba0485a99f1629cd6f9962b => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:23.223710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43641d59-8972-4bc9-a116-85fa02537adc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:23.293887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e50e70-a41b-4240-b599-ae0ddd875c5b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:23.320168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:23.332124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:23.364019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746440-bc6e-42ee-81fb-e5f37506c7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:23.364683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746440-bc6e-42ee-81fb-e5f37506c7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:23.364967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746440-bc6e-42ee-81fb-e5f37506c7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:23.365302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746440-bc6e-42ee-81fb-e5f37506c7a8 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:23.367741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a746440-bc6e-42ee-81fb-e5f37506c7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:23.369223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a746440-bc6e-42ee-81fb-e5f37506c7a8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:23.376344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 67/134] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:23 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:23.382401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746440-bc6e-42ee-81fb-e5f37506c7a8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:23.415389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43641d59-8972-4bc9-a116-85fa02537adc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:23.453793 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a5409-2c8c-4428-9155-affca415d157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:23.454468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a5409-2c8c-4428-9155-affca415d157 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:23.454804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a5409-2c8c-4428-9155-affca415d157 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:23.455087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a5409-2c8c-4428-9155-affca415d157 None admin] PATH_INFO: `/v3/projects/cefb9581523149d38c64301b2796ae2f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:23.456504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a5409-2c8c-4428-9155-affca415d157 None admin] RBAC: Authorizing `identity:get_project(project_id=cefb9581523149d38c64301b2796ae2f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:23.458024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a5409-2c8c-4428-9155-affca415d157 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:23.460506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 68/135] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:23 2022] GET /identity/v3/projects/cefb9581523149d38c64301b2796ae2f => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:23.470330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a5409-2c8c-4428-9155-affca415d157 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:23.595124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a5409-2c8c-4428-9155-affca415d157 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:23.616751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:23.631541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c57a3-a8e9-4319-87be-ae2d54b9c90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:23.632264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c57a3-a8e9-4319-87be-ae2d54b9c90b None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:23.632551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c57a3-a8e9-4319-87be-ae2d54b9c90b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:23.644594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c57a3-a8e9-4319-87be-ae2d54b9c90b None admin] PATH_INFO: `/v3/projects/cefb9581523149d38c64301b2796ae2f/users/a5948add9ba0485a99f1629cd6f9962b/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:23.644594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c57a3-a8e9-4319-87be-ae2d54b9c90b None admin] RBAC: Authorizing `identity:create_grant(project_id=cefb9581523149d38c64301b2796ae2f, user_id=a5948add9ba0485a99f1629cd6f9962b, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:23.644594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c57a3-a8e9-4319-87be-ae2d54b9c90b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:23.672038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746440-bc6e-42ee-81fb-e5f37506c7a8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:23.711833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e0f24-520f-4619-9d98-77164610016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:23.712519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e0f24-520f-4619-9d98-77164610016b None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:23.712809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e0f24-520f-4619-9d98-77164610016b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:23.713091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e0f24-520f-4619-9d98-77164610016b None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/users/a5948add9ba0485a99f1629cd6f9962b/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:23.721088 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e0f24-520f-4619-9d98-77164610016b None admin] RBAC: Authorizing `identity:create_grant(project_id=7fa1f9bede2243f0ba30c5f573dc695b, user_id=a5948add9ba0485a99f1629cd6f9962b, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:23.722557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e0f24-520f-4619-9d98-77164610016b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:23.736685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 69/136] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:04:23 2022] PUT /identity/v3/projects/cefb9581523149d38c64301b2796ae2f/users/a5948add9ba0485a99f1629cd6f9962b/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:23.795640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 68/137] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:04:23 2022] PUT /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/users/a5948add9ba0485a99f1629cd6f9962b/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:04:24.007839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0178720-88d4-4ce5-a9e1-3f959894918c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:24.008209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0178720-88d4-4ce5-a9e1-3f959894918c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:24.008403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0178720-88d4-4ce5-a9e1-3f959894918c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:24.010117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 70/138] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:24.015855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ed562-74ad-46ad-9d61-74ddfff6ca7f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:24.016151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ed562-74ad-46ad-9d61-74ddfff6ca7f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:24.016436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ed562-74ad-46ad-9d61-74ddfff6ca7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:24.039654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-742ed562-74ad-46ad-9d61-74ddfff6ca7f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:24.040941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-742ed562-74ad-46ad-9d61-74ddfff6ca7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:24.090233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:24.120265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 69/139] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:24.148461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0178720-88d4-4ce5-a9e1-3f959894918c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:24.288759 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0178720-88d4-4ce5-a9e1-3f959894918c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:24.329423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5514d-44bd-4e81-ad9b-1e56e85940f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:24.330422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5514d-44bd-4e81-ad9b-1e56e85940f1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:24.330857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5514d-44bd-4e81-ad9b-1e56e85940f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:24.331305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5514d-44bd-4e81-ad9b-1e56e85940f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:24.333238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5514d-44bd-4e81-ad9b-1e56e85940f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:24.335521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5514d-44bd-4e81-ad9b-1e56e85940f1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:24.337267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 71/140] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:24.342777 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ed562-74ad-46ad-9d61-74ddfff6ca7f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:24.460288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ed562-74ad-46ad-9d61-74ddfff6ca7f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:24.507273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a72cb8-250a-4946-a89f-b44800cacac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:24.507937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a72cb8-250a-4946-a89f-b44800cacac9 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:24.508222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a72cb8-250a-4946-a89f-b44800cacac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:24.508503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a72cb8-250a-4946-a89f-b44800cacac9 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:24.509211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a72cb8-250a-4946-a89f-b44800cacac9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:24.510674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a72cb8-250a-4946-a89f-b44800cacac9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:24.634463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 70/141] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:04:24 2022] POST /identity/v3/users => generated 283 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:24.740125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb69512-33f7-494d-8077-663264e97f56 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:24.740483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb69512-33f7-494d-8077-663264e97f56 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:24.740767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb69512-33f7-494d-8077-663264e97f56 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:24.742393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 72/142] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:24.747995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e6249-8d7d-4bf5-8307-b57757fcc6fb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:24.748599 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e6249-8d7d-4bf5-8307-b57757fcc6fb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:24.749070 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e6249-8d7d-4bf5-8307-b57757fcc6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:24.777081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-897e6249-8d7d-4bf5-8307-b57757fcc6fb None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:24.778748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-897e6249-8d7d-4bf5-8307-b57757fcc6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:24.813637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:24.872390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 71/143] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:24.909923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb69512-33f7-494d-8077-663264e97f56 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:24.965991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:25.043294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb69512-33f7-494d-8077-663264e97f56 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:25.096521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327120b-082c-4ed8-991e-10e0f458a883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:25.097195 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327120b-082c-4ed8-991e-10e0f458a883 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:25.097486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327120b-082c-4ed8-991e-10e0f458a883 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:25.097764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327120b-082c-4ed8-991e-10e0f458a883 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:25.110535 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327120b-082c-4ed8-991e-10e0f458a883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-3327120b-082c-4ed8-991e-10e0f458a883 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:25.116633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:25.117449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:25.118212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 73/144] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:25.123703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e6249-8d7d-4bf5-8307-b57757fcc6fb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:25.246074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e6249-8d7d-4bf5-8307-b57757fcc6fb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:25.284597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f4262-3f47-431b-a062-f55fffc7ed6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:25.285556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f4262-3f47-431b-a062-f55fffc7ed6d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:25.286027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f4262-3f47-431b-a062-f55fffc7ed6d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:25.286489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f4262-3f47-431b-a062-f55fffc7ed6d None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:25.287533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f4262-3f47-431b-a062-f55fffc7ed6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:25.288083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f4262-3f47-431b-a062-f55fffc7ed6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:25.289820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f4262-3f47-431b-a062-f55fffc7ed6d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:25.303435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 72/145] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:25.308688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327120b-082c-4ed8-991e-10e0f458a883 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:25.450498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:25.454203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327120b-082c-4ed8-991e-10e0f458a883 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:25.463444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433dc29-1aab-4616-b5bc-9c447571bf96 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:25.464070 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433dc29-1aab-4616-b5bc-9c447571bf96 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:25.464070 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433dc29-1aab-4616-b5bc-9c447571bf96 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:25.465473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 73/146] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:25.473151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe57a93-27bb-43b5-be0e-b7d401094467 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:25.478321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe57a93-27bb-43b5-be0e-b7d401094467 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:25.478321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe57a93-27bb-43b5-be0e-b7d401094467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:25.503473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ffe57a93-27bb-43b5-be0e-b7d401094467 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:25.505467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ffe57a93-27bb-43b5-be0e-b7d401094467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:25.517296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447514b-0d35-4803-93fa-78ea9cb700da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:25.518259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447514b-0d35-4803-93fa-78ea9cb700da None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:25.518713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447514b-0d35-4803-93fa-78ea9cb700da None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:25.519159 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447514b-0d35-4803-93fa-78ea9cb700da None admin] PATH_INFO: `/v3/users/a5948add9ba0485a99f1629cd6f9962b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:25.520531 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447514b-0d35-4803-93fa-78ea9cb700da None admin] RBAC: Authorizing `identity:get_user(user_id=a5948add9ba0485a99f1629cd6f9962b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:25.522245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447514b-0d35-4803-93fa-78ea9cb700da None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:25.526142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 74/147] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:25 2022] GET /identity/v3/users/a5948add9ba0485a99f1629cd6f9962b => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:25.532283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447514b-0d35-4803-93fa-78ea9cb700da None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:25.544126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:25.642514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 74/148] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:25.676074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447514b-0d35-4803-93fa-78ea9cb700da None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:25.676939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe57a93-27bb-43b5-be0e-b7d401094467 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:25.714284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ca2de-1cd9-48e4-a418-d8d5760ebc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:25.714900 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ca2de-1cd9-48e4-a418-d8d5760ebc0f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:25.715186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ca2de-1cd9-48e4-a418-d8d5760ebc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:25.715465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ca2de-1cd9-48e4-a418-d8d5760ebc0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:25.752702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:25.792336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ca2de-1cd9-48e4-a418-d8d5760ebc0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:25.793971 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ca2de-1cd9-48e4-a418-d8d5760ebc0f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:25.825681 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe57a93-27bb-43b5-be0e-b7d401094467 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:25.885109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d28e0-eb38-41ae-9283-d6d22f2020d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:25.885769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d28e0-eb38-41ae-9283-d6d22f2020d3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:25.886049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d28e0-eb38-41ae-9283-d6d22f2020d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:25.886357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d28e0-eb38-41ae-9283-d6d22f2020d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:25.902078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 75/149] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:25.908091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ca2de-1cd9-48e4-a418-d8d5760ebc0f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:25.911855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d28e0-eb38-41ae-9283-d6d22f2020d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9e4d28e0-eb38-41ae-9283-d6d22f2020d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:25.922216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:25.923153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:25.923958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 75/150] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:25.931806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d28e0-eb38-41ae-9283-d6d22f2020d3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:25.956704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:26.005786 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:26.057884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ca2de-1cd9-48e4-a418-d8d5760ebc0f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:26.100383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00afea-f8ba-43a2-a581-f336430ca9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:26.101023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00afea-f8ba-43a2-a581-f336430ca9bc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:26.101304 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00afea-f8ba-43a2-a581-f336430ca9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:26.101578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00afea-f8ba-43a2-a581-f336430ca9bc None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:26.102512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d28e0-eb38-41ae-9283-d6d22f2020d3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:26.102735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00afea-f8ba-43a2-a581-f336430ca9bc None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:26.106746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00afea-f8ba-43a2-a581-f336430ca9bc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:26.106746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 76/151] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:25 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:26.112266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00afea-f8ba-43a2-a581-f336430ca9bc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:26.165589 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcef04-28cb-422e-918b-3af1c75dc23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:26.166619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcef04-28cb-422e-918b-3af1c75dc23c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:26.167089 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcef04-28cb-422e-918b-3af1c75dc23c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:26.167505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcef04-28cb-422e-918b-3af1c75dc23c None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:26.168597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcef04-28cb-422e-918b-3af1c75dc23c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:26.169043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcef04-28cb-422e-918b-3af1c75dc23c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:26.170649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcef04-28cb-422e-918b-3af1c75dc23c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:26.187464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 76/152] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:26.193653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcef04-28cb-422e-918b-3af1c75dc23c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:26.274057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00afea-f8ba-43a2-a581-f336430ca9bc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:26.300252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:26.326531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcef04-28cb-422e-918b-3af1c75dc23c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:26.363541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2467ff-e811-4d3e-8764-4010a7c13786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:26.366831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2467ff-e811-4d3e-8764-4010a7c13786 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:26.366943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8555f-68d2-4bbd-acdc-e213867da850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:26.367056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2467ff-e811-4d3e-8764-4010a7c13786 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:26.367056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2467ff-e811-4d3e-8764-4010a7c13786 None admin] PATH_INFO: `/v3/users/bb37d398d5fd456aa46fd2b5f5576815` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:26.367178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8555f-68d2-4bbd-acdc-e213867da850 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:26.367178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8555f-68d2-4bbd-acdc-e213867da850 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:26.367178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8555f-68d2-4bbd-acdc-e213867da850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:26.367350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2467ff-e811-4d3e-8764-4010a7c13786 None admin] RBAC: Authorizing `identity:get_user(user_id=bb37d398d5fd456aa46fd2b5f5576815)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:26.367415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8555f-68d2-4bbd-acdc-e213867da850 None admin] RBAC: Adding query filter params (role.id=b5b241afd53c456e9a9fa3033eea74ad, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b, user.id=a5948add9ba0485a99f1629cd6f9962b) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:26.367415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8555f-68d2-4bbd-acdc-e213867da850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:26.368893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2467ff-e811-4d3e-8764-4010a7c13786 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:26.369006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8555f-68d2-4bbd-acdc-e213867da850 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:26.369909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 77/153] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:26 2022] GET /identity/v3/users/bb37d398d5fd456aa46fd2b5f5576815 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:26.372103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:26.374008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfc09-81a7-4e96-84d3-b77ac99bb229 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:26.374298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfc09-81a7-4e96-84d3-b77ac99bb229 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:26.374620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebfc09-81a7-4e96-84d3-b77ac99bb229 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:26.376036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 78/154] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:26.378431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebfc09-81a7-4e96-84d3-b77ac99bb229 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:26.382693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 77/155] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:26 2022] GET /identity/v3/role_assignments?user.id=a5948add9ba0485a99f1629cd6f9962b&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=b5b241afd53c456e9a9fa3033eea74ad => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:26.383999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061942-092c-46d3-82c4-730ff46f7e4a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:26.384290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061942-092c-46d3-82c4-730ff46f7e4a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:26.387846 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061942-092c-46d3-82c4-730ff46f7e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:26.421395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a9061942-092c-46d3-82c4-730ff46f7e4a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:26.422621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a9061942-092c-46d3-82c4-730ff46f7e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:26.519945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 78/156] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:26.542481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061942-092c-46d3-82c4-730ff46f7e4a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:26.572928 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebfc09-81a7-4e96-84d3-b77ac99bb229 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:26.628763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987a606-e0e2-4d4f-938e-bbe134e57e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:26.629416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987a606-e0e2-4d4f-938e-bbe134e57e72 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:26.629702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987a606-e0e2-4d4f-938e-bbe134e57e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:26.629981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987a606-e0e2-4d4f-938e-bbe134e57e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:26.658930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061942-092c-46d3-82c4-730ff46f7e4a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:26.661027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:26.688254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:26.701376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a701c2f-98aa-498e-bcac-59b61fa22f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:26.702038 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701c2f-98aa-498e-bcac-59b61fa22f31 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:26.702319 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701c2f-98aa-498e-bcac-59b61fa22f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:26.702591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701c2f-98aa-498e-bcac-59b61fa22f31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:26.725180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a701c2f-98aa-498e-bcac-59b61fa22f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0a701c2f-98aa-498e-bcac-59b61fa22f31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:26.730140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:26.731053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:26.731859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:26.736718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 79/157] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:26.742026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a701c2f-98aa-498e-bcac-59b61fa22f31 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:26.779416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987a606-e0e2-4d4f-938e-bbe134e57e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:26.780932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987a606-e0e2-4d4f-938e-bbe134e57e72 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:26.904114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 79/158] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:26.910501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987a606-e0e2-4d4f-938e-bbe134e57e72 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:26.916445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a701c2f-98aa-498e-bcac-59b61fa22f31 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:26.968640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:26.980508 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a06255-0adc-445a-b250-801f28d1f28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:26.981488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a06255-0adc-445a-b250-801f28d1f28c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:26.981876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a06255-0adc-445a-b250-801f28d1f28c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:26.982258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a06255-0adc-445a-b250-801f28d1f28c None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:26.983093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a06255-0adc-445a-b250-801f28d1f28c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:26.983503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a06255-0adc-445a-b250-801f28d1f28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:26.985054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a06255-0adc-445a-b250-801f28d1f28c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:27.000372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 80/159] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:27.006684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a06255-0adc-445a-b250-801f28d1f28c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:27.051272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987a606-e0e2-4d4f-938e-bbe134e57e72 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:27.079185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:27.111268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8704beb-0b09-46d5-8871-c42c779fee85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:27.111969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8704beb-0b09-46d5-8871-c42c779fee85 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:27.112222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8704beb-0b09-46d5-8871-c42c779fee85 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:27.112537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8704beb-0b09-46d5-8871-c42c779fee85 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:27.116085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8704beb-0b09-46d5-8871-c42c779fee85 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:27.117699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8704beb-0b09-46d5-8871-c42c779fee85 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:27.119525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 80/160] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:26 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:27.137739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a06255-0adc-445a-b250-801f28d1f28c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:27.145781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8704beb-0b09-46d5-8871-c42c779fee85 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:27.176067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748778a-cfc9-4709-896d-18721fd05341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:27.177024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748778a-cfc9-4709-896d-18721fd05341 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:27.178083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748778a-cfc9-4709-896d-18721fd05341 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:27.178498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748778a-cfc9-4709-896d-18721fd05341 None admin] PATH_INFO: `/v3/users/a05cd95cd10645a392f5bcfee57786e7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:27.179909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748778a-cfc9-4709-896d-18721fd05341 None admin] RBAC: Authorizing `identity:get_user(user_id=a05cd95cd10645a392f5bcfee57786e7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:27.181519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748778a-cfc9-4709-896d-18721fd05341 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:27.183405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 81/161] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:27 2022] GET /identity/v3/users/a05cd95cd10645a392f5bcfee57786e7 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:27.188973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748778a-cfc9-4709-896d-18721fd05341 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:27.332770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8704beb-0b09-46d5-8871-c42c779fee85 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:27.351816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:27.392051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748778a-cfc9-4709-896d-18721fd05341 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:27.411242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a233e3-d4b2-4c06-be6b-dbff3e0fe939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:27.412420 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a233e3-d4b2-4c06-be6b-dbff3e0fe939 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:27.412718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a233e3-d4b2-4c06-be6b-dbff3e0fe939 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:27.412993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a233e3-d4b2-4c06-be6b-dbff3e0fe939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:27.413788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a233e3-d4b2-4c06-be6b-dbff3e0fe939 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b, user.id=bb37d398d5fd456aa46fd2b5f5576815) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:27.414100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a233e3-d4b2-4c06-be6b-dbff3e0fe939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:27.415560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a233e3-d4b2-4c06-be6b-dbff3e0fe939 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:27.424258 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 81/162] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:27 2022] GET /identity/v3/role_assignments?user.id=bb37d398d5fd456aa46fd2b5f5576815&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:27.424787 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:27.425505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152404c2-6f82-4d2d-91ef-a74dccb316ed None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:27.425795 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152404c2-6f82-4d2d-91ef-a74dccb316ed None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:27.426209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152404c2-6f82-4d2d-91ef-a74dccb316ed None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:27.427516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 82/163] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:27.441912 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e644a9-4ffd-4d00-af6f-e2a2b2880546 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:27.442251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e644a9-4ffd-4d00-af6f-e2a2b2880546 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:27.442536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e644a9-4ffd-4d00-af6f-e2a2b2880546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:27.454635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05068b5c-01f1-4011-b88f-4f7be3f1d6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:27.455311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05068b5c-01f1-4011-b88f-4f7be3f1d6ec None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:27.455591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05068b5c-01f1-4011-b88f-4f7be3f1d6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:27.455905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05068b5c-01f1-4011-b88f-4f7be3f1d6ec None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:27.458001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05068b5c-01f1-4011-b88f-4f7be3f1d6ec None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:27.460391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05068b5c-01f1-4011-b88f-4f7be3f1d6ec None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:27.462183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 82/164] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:27 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:27.468561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05068b5c-01f1-4011-b88f-4f7be3f1d6ec None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:27.484884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-81e644a9-4ffd-4d00-af6f-e2a2b2880546 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:27.486112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-81e644a9-4ffd-4d00-af6f-e2a2b2880546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:27.649763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05068b5c-01f1-4011-b88f-4f7be3f1d6ec None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:27.709811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3192f72-663c-4fe8-9d6a-afcb183a6620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:27.710962 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3192f72-663c-4fe8-9d6a-afcb183a6620 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:27.711437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3192f72-663c-4fe8-9d6a-afcb183a6620 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:27.711951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3192f72-663c-4fe8-9d6a-afcb183a6620 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/users/a05cd95cd10645a392f5bcfee57786e7/roles/324a803dd87d4027b73d871fe46ea228` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:27.714275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3192f72-663c-4fe8-9d6a-afcb183a6620 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fa1f9bede2243f0ba30c5f573dc695b, user_id=a05cd95cd10645a392f5bcfee57786e7, role_id=324a803dd87d4027b73d871fe46ea228)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:27.716023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3192f72-663c-4fe8-9d6a-afcb183a6620 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:27.721761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 83/165] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:27.723156 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1aef78-0f55-4234-a108-b5b31d5f0c06 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:27.723449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1aef78-0f55-4234-a108-b5b31d5f0c06 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:27.723725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1aef78-0f55-4234-a108-b5b31d5f0c06 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:27.725582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 84/166] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:27.730890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a483d53-d820-4d88-b7c4-06092fbe4320 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:27.731175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a483d53-d820-4d88-b7c4-06092fbe4320 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:27.731452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a483d53-d820-4d88-b7c4-06092fbe4320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:27.756831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4a483d53-d820-4d88-b7c4-06092fbe4320 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:27.758016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4a483d53-d820-4d88-b7c4-06092fbe4320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:27.798517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 83/167] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:04:27 2022] PUT /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/users/a05cd95cd10645a392f5bcfee57786e7/roles/324a803dd87d4027b73d871fe46ea228 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:27.799711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3192f72-663c-4fe8-9d6a-afcb183a6620 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:27.842651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 85/168] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:27.868642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a483d53-d820-4d88-b7c4-06092fbe4320 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:27.895976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:27.907522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:27.975767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3192f72-663c-4fe8-9d6a-afcb183a6620 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:28.048561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922501c-9b9a-4be3-beed-ed85e7a2f681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:28.049234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922501c-9b9a-4be3-beed-ed85e7a2f681 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:28.049515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922501c-9b9a-4be3-beed-ed85e7a2f681 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:28.049809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922501c-9b9a-4be3-beed-ed85e7a2f681 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:28.066831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a483d53-d820-4d88-b7c4-06092fbe4320 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:28.070100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922501c-9b9a-4be3-beed-ed85e7a2f681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b922501c-9b9a-4be3-beed-ed85e7a2f681 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:28.080208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:28.081365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:28.082147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 84/169] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:28.088460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922501c-9b9a-4be3-beed-ed85e7a2f681 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:28.121891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fb2d9-e580-4868-9d19-3e1fbc1d2f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:28.122563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fb2d9-e580-4868-9d19-3e1fbc1d2f2c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:28.122844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fb2d9-e580-4868-9d19-3e1fbc1d2f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:28.123120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fb2d9-e580-4868-9d19-3e1fbc1d2f2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:28.148464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fb2d9-e580-4868-9d19-3e1fbc1d2f2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-7b3fb2d9-e580-4868-9d19-3e1fbc1d2f2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:28.153628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:28.154501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:28.155390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 86/170] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:28.162987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fb2d9-e580-4868-9d19-3e1fbc1d2f2c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:28.250017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922501c-9b9a-4be3-beed-ed85e7a2f681 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:28.258864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:28.305217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42985eae-cae2-4267-b4ec-b5062d986314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:28.305870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42985eae-cae2-4267-b4ec-b5062d986314 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:28.306158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42985eae-cae2-4267-b4ec-b5062d986314 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:28.306438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42985eae-cae2-4267-b4ec-b5062d986314 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:28.307176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42985eae-cae2-4267-b4ec-b5062d986314 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:28.307485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42985eae-cae2-4267-b4ec-b5062d986314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:28.309196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42985eae-cae2-4267-b4ec-b5062d986314 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:28.321943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 85/171] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:28.331858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42985eae-cae2-4267-b4ec-b5062d986314 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:28.382196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:28.395911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fb2d9-e580-4868-9d19-3e1fbc1d2f2c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:28.497090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8eca9b-62b6-4d6a-98e6-3b15253418a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:28.497730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8eca9b-62b6-4d6a-98e6-3b15253418a1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:28.498019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8eca9b-62b6-4d6a-98e6-3b15253418a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:28.498295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8eca9b-62b6-4d6a-98e6-3b15253418a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:28.499028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8eca9b-62b6-4d6a-98e6-3b15253418a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:28.499339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8eca9b-62b6-4d6a-98e6-3b15253418a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:28.500825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8eca9b-62b6-4d6a-98e6-3b15253418a1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:28.521729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42985eae-cae2-4267-b4ec-b5062d986314 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:28.521904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 87/172] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:28.527560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8eca9b-62b6-4d6a-98e6-3b15253418a1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:28.613834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d913c-ad8d-4a3e-af4b-4e44b24a4e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:28.614497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d913c-ad8d-4a3e-af4b-4e44b24a4e5a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:28.614784 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d913c-ad8d-4a3e-af4b-4e44b24a4e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:28.615591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d913c-ad8d-4a3e-af4b-4e44b24a4e5a None admin] PATH_INFO: `/v3/users/a5948add9ba0485a99f1629cd6f9962b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:28.620536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d913c-ad8d-4a3e-af4b-4e44b24a4e5a None admin] RBAC: Authorizing `identity:get_user(user_id=a5948add9ba0485a99f1629cd6f9962b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:28.622031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d913c-ad8d-4a3e-af4b-4e44b24a4e5a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:28.623865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 86/173] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:28 2022] GET /identity/v3/users/a5948add9ba0485a99f1629cd6f9962b => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:28.629198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d913c-ad8d-4a3e-af4b-4e44b24a4e5a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:28.729992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8eca9b-62b6-4d6a-98e6-3b15253418a1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:28.733158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:28.806570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79124c86-2a8f-4d77-b94f-4629334d4b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:28.807141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79124c86-2a8f-4d77-b94f-4629334d4b5d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:28.807428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79124c86-2a8f-4d77-b94f-4629334d4b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:28.807750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79124c86-2a8f-4d77-b94f-4629334d4b5d None admin] PATH_INFO: `/v3/users/a5948add9ba0485a99f1629cd6f9962b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:28.812555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79124c86-2a8f-4d77-b94f-4629334d4b5d None admin] RBAC: Authorizing `identity:get_user(user_id=a5948add9ba0485a99f1629cd6f9962b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:28.814015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79124c86-2a8f-4d77-b94f-4629334d4b5d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:28.815978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 88/174] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:28 2022] GET /identity/v3/users/a5948add9ba0485a99f1629cd6f9962b => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:28.822030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79124c86-2a8f-4d77-b94f-4629334d4b5d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:28.876158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:28.886032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d913c-ad8d-4a3e-af4b-4e44b24a4e5a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:28.959740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331eef3-7b16-4528-aa68-7381610b63ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:28.960417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331eef3-7b16-4528-aa68-7381610b63ca None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:28.960698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331eef3-7b16-4528-aa68-7381610b63ca None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:28.960978 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331eef3-7b16-4528-aa68-7381610b63ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.025035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79124c86-2a8f-4d77-b94f-4629334d4b5d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:29.078610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9331eef3-7b16-4528-aa68-7381610b63ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:29.080172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9331eef3-7b16-4528-aa68-7381610b63ca None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:29.128186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:29.148516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3bf70-eff2-4b3a-9c05-f7b349b2bbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:29.149154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3bf70-eff2-4b3a-9c05-f7b349b2bbcf None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.149442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3bf70-eff2-4b3a-9c05-f7b349b2bbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.149721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3bf70-eff2-4b3a-9c05-f7b349b2bbcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.184620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 87/175] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:29.186398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cf002-64c5-4863-9ebc-1827aefe8a33 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.186700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cf002-64c5-4863-9ebc-1827aefe8a33 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.187467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cf002-64c5-4863-9ebc-1827aefe8a33 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.188881 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 88/176] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:29.193709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cf002-64c5-4863-9ebc-1827aefe8a33 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:29.301034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cf002-64c5-4863-9ebc-1827aefe8a33 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:29.332718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3bf70-eff2-4b3a-9c05-f7b349b2bbcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:29.335016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3bf70-eff2-4b3a-9c05-f7b349b2bbcf None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:29.337207 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e930d74-642f-44bc-9009-22e2bc521021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:29.337855 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e930d74-642f-44bc-9009-22e2bc521021 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.338173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e930d74-642f-44bc-9009-22e2bc521021 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.338457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e930d74-642f-44bc-9009-22e2bc521021 None admin] PATH_INFO: `/v3/projects/cefb9581523149d38c64301b2796ae2f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.339653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e930d74-642f-44bc-9009-22e2bc521021 None admin] RBAC: Authorizing `identity:get_project(project_id=cefb9581523149d38c64301b2796ae2f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:29.341185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e930d74-642f-44bc-9009-22e2bc521021 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:29.342861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 89/177] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:29 2022] GET /identity/v3/projects/cefb9581523149d38c64301b2796ae2f => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:29.343999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a633bb-78e6-47d2-baae-91e8c4f7c711 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.344346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a633bb-78e6-47d2-baae-91e8c4f7c711 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.344622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a633bb-78e6-47d2-baae-91e8c4f7c711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.352329 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:29.362439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:29.363915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-65a633bb-78e6-47d2-baae-91e8c4f7c711 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:29.365178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-65a633bb-78e6-47d2-baae-91e8c4f7c711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:29.434729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 90/178] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:29.436060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52585f64-b788-4a15-90e2-6b69312d829b None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.436419 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52585f64-b788-4a15-90e2-6b69312d829b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.436766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52585f64-b788-4a15-90e2-6b69312d829b None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.442385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 91/179] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:29.442385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52585f64-b788-4a15-90e2-6b69312d829b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:29.446916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 89/180] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 01:04:29.452970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07943c7-138b-4c00-a929-a1431d7acae0 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.453265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07943c7-138b-4c00-a929-a1431d7acae0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.453549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07943c7-138b-4c00-a929-a1431d7acae0 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.455015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 90/181] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:29.460956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09310523-21ec-4bd3-b189-a73485b8cf2e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.461255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09310523-21ec-4bd3-b189-a73485b8cf2e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.461523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09310523-21ec-4bd3-b189-a73485b8cf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.480752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-09310523-21ec-4bd3-b189-a73485b8cf2e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:29.481964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-09310523-21ec-4bd3-b189-a73485b8cf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:29.555514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 91/182] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:29.556658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09310523-21ec-4bd3-b189-a73485b8cf2e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:29.567518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52585f64-b788-4a15-90e2-6b69312d829b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:29.608471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae1da0-d76c-4284-8f72-897cb1a7245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:29.609169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae1da0-d76c-4284-8f72-897cb1a7245c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.609606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae1da0-d76c-4284-8f72-897cb1a7245c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.609763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae1da0-d76c-4284-8f72-897cb1a7245c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.610591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae1da0-d76c-4284-8f72-897cb1a7245c None admin] RBAC: Adding query filter params (role.id=512a743b8ca748f8b793681748be7906, scope.project.id=cefb9581523149d38c64301b2796ae2f, user.id=a5948add9ba0485a99f1629cd6f9962b) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:29.610910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae1da0-d76c-4284-8f72-897cb1a7245c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:29.612375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae1da0-d76c-4284-8f72-897cb1a7245c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:29.614548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:29.618992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 92/183] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:29 2022] GET /identity/v3/role_assignments?user.id=a5948add9ba0485a99f1629cd6f9962b&scope.project.id=cefb9581523149d38c64301b2796ae2f&role.id=512a743b8ca748f8b793681748be7906 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:29.621151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2148210-5f1e-4bad-914b-3804c32b6b58 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.622135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2148210-5f1e-4bad-914b-3804c32b6b58 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.622601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2148210-5f1e-4bad-914b-3804c32b6b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.638169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:29.650901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f2148210-5f1e-4bad-914b-3804c32b6b58 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:29.653462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f2148210-5f1e-4bad-914b-3804c32b6b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:29.691348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09310523-21ec-4bd3-b189-a73485b8cf2e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:29.743605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91603ddd-96a5-48b5-a5da-369b2c9b3a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:29.745515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91603ddd-96a5-48b5-a5da-369b2c9b3a58 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.746019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91603ddd-96a5-48b5-a5da-369b2c9b3a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.746741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91603ddd-96a5-48b5-a5da-369b2c9b3a58 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.748331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91603ddd-96a5-48b5-a5da-369b2c9b3a58 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:29.750544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91603ddd-96a5-48b5-a5da-369b2c9b3a58 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:29.752984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 92/184] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:29 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:29.757919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91603ddd-96a5-48b5-a5da-369b2c9b3a58 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:29.765524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 93/185] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:29.767060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2148210-5f1e-4bad-914b-3804c32b6b58 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:29.895499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91603ddd-96a5-48b5-a5da-369b2c9b3a58 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:29.901707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2148210-5f1e-4bad-914b-3804c32b6b58 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:29.919471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:29.931104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cf3f5-48dd-4393-a252-f00a3fe06f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:29.931749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cf3f5-48dd-4393-a252-f00a3fe06f5e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.932059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cf3f5-48dd-4393-a252-f00a3fe06f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.932331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cf3f5-48dd-4393-a252-f00a3fe06f5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.935221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:29.942593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cf3f5-48dd-4393-a252-f00a3fe06f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:29.945099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f2bac-1334-4d5b-9ec3-0c96ec184b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:29.945753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f2bac-1334-4d5b-9ec3-0c96ec184b1a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:29.946035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f2bac-1334-4d5b-9ec3-0c96ec184b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:29.946313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f2bac-1334-4d5b-9ec3-0c96ec184b1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:29.947496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f2bac-1334-4d5b-9ec3-0c96ec184b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0c8cf3f5-48dd-4393-a252-f00a3fe06f5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:29.948252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:29.949291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:29.950573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:29.951475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f2bac-1334-4d5b-9ec3-0c96ec184b1a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:29.951475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 94/186] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:29.952064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f2bac-1334-4d5b-9ec3-0c96ec184b1a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:29.962357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 93/187] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:29.962357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cf3f5-48dd-4393-a252-f00a3fe06f5e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:30.058630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f2bac-1334-4d5b-9ec3-0c96ec184b1a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:30.063948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cf3f5-48dd-4393-a252-f00a3fe06f5e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:30.100391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0004d-56d6-4833-8aa5-d61cbcb9f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:30.101066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0004d-56d6-4833-8aa5-d61cbcb9f127 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.101343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0004d-56d6-4833-8aa5-d61cbcb9f127 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.101622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0004d-56d6-4833-8aa5-d61cbcb9f127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.102404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0004d-56d6-4833-8aa5-d61cbcb9f127 None admin] RBAC: Adding query filter params (role.id=512a743b8ca748f8b793681748be7906, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b, user.id=a5948add9ba0485a99f1629cd6f9962b) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:30.102715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0004d-56d6-4833-8aa5-d61cbcb9f127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:30.104177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0004d-56d6-4833-8aa5-d61cbcb9f127 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:30.107209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf08843-66a9-41aa-a06f-5f0dfe17e1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:30.108534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf08843-66a9-41aa-a06f-5f0dfe17e1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.108826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf08843-66a9-41aa-a06f-5f0dfe17e1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.109106 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf08843-66a9-41aa-a06f-5f0dfe17e1f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.120214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 95/188] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:29 2022] GET /identity/v3/role_assignments?user.id=a5948add9ba0485a99f1629cd6f9962b&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=512a743b8ca748f8b793681748be7906 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:30.120214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0004d-56d6-4833-8aa5-d61cbcb9f127 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:30.128888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf08843-66a9-41aa-a06f-5f0dfe17e1f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bbf08843-66a9-41aa-a06f-5f0dfe17e1f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:30.135077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:30.136161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:30.137367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 94/189] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:30.142104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf08843-66a9-41aa-a06f-5f0dfe17e1f5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:30.192533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:30.199728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:30.258352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf08843-66a9-41aa-a06f-5f0dfe17e1f5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:30.259274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0004d-56d6-4833-8aa5-d61cbcb9f127 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:30.305870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125869f-9685-4c9e-a93b-a46d62d0e91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:30.306784 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125869f-9685-4c9e-a93b-a46d62d0e91d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.307270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125869f-9685-4c9e-a93b-a46d62d0e91d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.307506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007447e9-d459-4c79-b62d-1e6bd5aba6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:30.307636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125869f-9685-4c9e-a93b-a46d62d0e91d None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.308185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007447e9-d459-4c79-b62d-1e6bd5aba6e6 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.308282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125869f-9685-4c9e-a93b-a46d62d0e91d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:30.308474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007447e9-d459-4c79-b62d-1e6bd5aba6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.308568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125869f-9685-4c9e-a93b-a46d62d0e91d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:30.308752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007447e9-d459-4c79-b62d-1e6bd5aba6e6 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.309452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007447e9-d459-4c79-b62d-1e6bd5aba6e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:30.309927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125869f-9685-4c9e-a93b-a46d62d0e91d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:30.310906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007447e9-d459-4c79-b62d-1e6bd5aba6e6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:30.319506 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 95/190] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:30.320374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125869f-9685-4c9e-a93b-a46d62d0e91d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:30.413203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 96/191] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:04:30 2022] POST /identity/v3/users => generated 297 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:30.414074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007447e9-d459-4c79-b62d-1e6bd5aba6e6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:30.428851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125869f-9685-4c9e-a93b-a46d62d0e91d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:30.432087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:30.469899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4a2c4-d2d4-4260-857a-1d86b702904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:30.470554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4a2c4-d2d4-4260-857a-1d86b702904a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.470886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4a2c4-d2d4-4260-857a-1d86b702904a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.471213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4a2c4-d2d4-4260-857a-1d86b702904a None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.472003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4a2c4-d2d4-4260-857a-1d86b702904a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:30.472435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4a2c4-d2d4-4260-857a-1d86b702904a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:30.474369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4a2c4-d2d4-4260-857a-1d86b702904a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:30.485307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 96/192] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:30.486670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487259-ec80-430e-abaa-faeb22303895 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.486981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487259-ec80-430e-abaa-faeb22303895 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.487246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487259-ec80-430e-abaa-faeb22303895 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.493040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 97/193] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:30.493040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487259-ec80-430e-abaa-faeb22303895 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:30.535340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007447e9-d459-4c79-b62d-1e6bd5aba6e6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:30.554461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:30.581972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc359134-e411-4d4f-ae52-6261c3590ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:30.583287 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc359134-e411-4d4f-ae52-6261c3590ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.584277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc359134-e411-4d4f-ae52-6261c3590ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.585968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc359134-e411-4d4f-ae52-6261c3590ea0 None admin] PATH_INFO: `/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.608589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc359134-e411-4d4f-ae52-6261c3590ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=e558a5118d9149a78c5bc1a4f4dbb7b7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:30.610093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc359134-e411-4d4f-ae52-6261c3590ea0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:30.613816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 97/194] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:30 2022] GET /identity/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:30.613816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf13305-c497-4a66-ae8e-bc6777e9892b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.613816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf13305-c497-4a66-ae8e-bc6777e9892b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.613816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf13305-c497-4a66-ae8e-bc6777e9892b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.618688 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487259-ec80-430e-abaa-faeb22303895 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:30.636495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-daf13305-c497-4a66-ae8e-bc6777e9892b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:30.638529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-daf13305-c497-4a66-ae8e-bc6777e9892b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:30.662680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e816a62-b125-4344-b93e-0f11bf8c7370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:30.663348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e816a62-b125-4344-b93e-0f11bf8c7370 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.663630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e816a62-b125-4344-b93e-0f11bf8c7370 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.663943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e816a62-b125-4344-b93e-0f11bf8c7370 None admin] PATH_INFO: `/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.665152 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e816a62-b125-4344-b93e-0f11bf8c7370 None admin] RBAC: Authorizing `identity:get_user(user_id=e558a5118d9149a78c5bc1a4f4dbb7b7)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:30.666620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e816a62-b125-4344-b93e-0f11bf8c7370 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:30.669205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 98/195] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:30 2022] GET /identity/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:30.669981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e816a62-b125-4344-b93e-0f11bf8c7370 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:30.720361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:30.721444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 98/196] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:30.722315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf13305-c497-4a66-ae8e-bc6777e9892b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:30.813612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e816a62-b125-4344-b93e-0f11bf8c7370 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:30.837124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf13305-c497-4a66-ae8e-bc6777e9892b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:30.840307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:30.862004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b4944-e2ff-469d-827c-ee72a2bdd560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:30.863158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b4944-e2ff-469d-827c-ee72a2bdd560 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.864027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b4944-e2ff-469d-827c-ee72a2bdd560 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.864784 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b4944-e2ff-469d-827c-ee72a2bdd560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.874442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7efb6d-c384-49d6-a095-0d5fe9713523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:30.875148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7efb6d-c384-49d6-a095-0d5fe9713523 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:30.875720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7efb6d-c384-49d6-a095-0d5fe9713523 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:30.875720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7efb6d-c384-49d6-a095-0d5fe9713523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:30.949424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7efb6d-c384-49d6-a095-0d5fe9713523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:30.951192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7efb6d-c384-49d6-a095-0d5fe9713523 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:30.951572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b4944-e2ff-469d-827c-ee72a2bdd560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:30.953467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b4944-e2ff-469d-827c-ee72a2bdd560 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:31.000348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:31.075256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 99/197] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:31.077015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b4944-e2ff-469d-827c-ee72a2bdd560 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:31.077148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 99/198] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:31.085354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7efb6d-c384-49d6-a095-0d5fe9713523 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:31.102344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:31.191440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b4944-e2ff-469d-827c-ee72a2bdd560 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:31.208528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7efb6d-c384-49d6-a095-0d5fe9713523 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:31.240148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca078d7-24ac-4305-964e-b8961ee8bc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:31.241844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca078d7-24ac-4305-964e-b8961ee8bc9d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:31.242344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca078d7-24ac-4305-964e-b8961ee8bc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:31.242825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca078d7-24ac-4305-964e-b8961ee8bc9d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:31.244938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:31.245749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae8acc-0832-4721-9c1a-9a5e24d36d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:31.246164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae8acc-0832-4721-9c1a-9a5e24d36d79 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:31.246430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae8acc-0832-4721-9c1a-9a5e24d36d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:31.246704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae8acc-0832-4721-9c1a-9a5e24d36d79 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:31.247952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae8acc-0832-4721-9c1a-9a5e24d36d79 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:31.249408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae8acc-0832-4721-9c1a-9a5e24d36d79 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:31.251284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 100/199] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:31 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:31.257082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae8acc-0832-4721-9c1a-9a5e24d36d79 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:31.264023 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca078d7-24ac-4305-964e-b8961ee8bc9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bca078d7-24ac-4305-964e-b8961ee8bc9d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:31.271128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:31.272650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:31.273462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:31.276257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 100/200] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:31.281486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca078d7-24ac-4305-964e-b8961ee8bc9d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:31.341606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:31.373081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae8acc-0832-4721-9c1a-9a5e24d36d79 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:31.411667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca078d7-24ac-4305-964e-b8961ee8bc9d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:31.414837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7657997-f1f3-4fb9-87df-6e8e30d9e5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:31.415487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7657997-f1f3-4fb9-87df-6e8e30d9e5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:31.415771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7657997-f1f3-4fb9-87df-6e8e30d9e5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:31.416892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7657997-f1f3-4fb9-87df-6e8e30d9e5c1 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:31.418071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7657997-f1f3-4fb9-87df-6e8e30d9e5c1 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:31.419588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7657997-f1f3-4fb9-87df-6e8e30d9e5c1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:31.421797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 101/201] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:31 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:31.422631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7657997-f1f3-4fb9-87df-6e8e30d9e5c1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:31.454991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f1bb3-6b28-49a9-950e-d21f5ed637e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:31.455713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f1bb3-6b28-49a9-950e-d21f5ed637e2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:31.456030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f1bb3-6b28-49a9-950e-d21f5ed637e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:31.456314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f1bb3-6b28-49a9-950e-d21f5ed637e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:31.457112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f1bb3-6b28-49a9-950e-d21f5ed637e2 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=3059810e56d143ab91728e2220010d53, user.id=e558a5118d9149a78c5bc1a4f4dbb7b7) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:31.457530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f1bb3-6b28-49a9-950e-d21f5ed637e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:31.459707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f1bb3-6b28-49a9-950e-d21f5ed637e2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:31.465499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 101/202] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:31 2022] GET /identity/v3/role_assignments?user.id=e558a5118d9149a78c5bc1a4f4dbb7b7&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:31.466288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f1bb3-6b28-49a9-950e-d21f5ed637e2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:31.536318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:31.562397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7657997-f1f3-4fb9-87df-6e8e30d9e5c1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:31.618475 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f1bb3-6b28-49a9-950e-d21f5ed637e2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:31.641458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272523a-c590-45d3-851a-af94ee974b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:31.642109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272523a-c590-45d3-851a-af94ee974b92 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:31.642393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272523a-c590-45d3-851a-af94ee974b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:31.642667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272523a-c590-45d3-851a-af94ee974b92 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:31.643408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272523a-c590-45d3-851a-af94ee974b92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:31.643717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272523a-c590-45d3-851a-af94ee974b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:31.645154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272523a-c590-45d3-851a-af94ee974b92 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:31.666781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 102/203] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:31.673129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272523a-c590-45d3-851a-af94ee974b92 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:31.676165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:31.700175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a386f-83d6-4e60-b7c9-ba0e25178563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:31.701300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a386f-83d6-4e60-b7c9-ba0e25178563 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:31.705733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a386f-83d6-4e60-b7c9-ba0e25178563 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:31.705733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a386f-83d6-4e60-b7c9-ba0e25178563 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:31.705733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a386f-83d6-4e60-b7c9-ba0e25178563 None admin] RBAC: Adding query filter params (role.id=b5b241afd53c456e9a9fa3033eea74ad, scope.project.id=3059810e56d143ab91728e2220010d53, user.id=e558a5118d9149a78c5bc1a4f4dbb7b7) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:31.705733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a386f-83d6-4e60-b7c9-ba0e25178563 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:31.705733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a386f-83d6-4e60-b7c9-ba0e25178563 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:31.711752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 102/204] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:31 2022] GET /identity/v3/role_assignments?user.id=e558a5118d9149a78c5bc1a4f4dbb7b7&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=b5b241afd53c456e9a9fa3033eea74ad => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:31.816290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272523a-c590-45d3-851a-af94ee974b92 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:31.859506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff6f12-a449-4e76-935d-d161fed16dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:31.861014 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff6f12-a449-4e76-935d-d161fed16dfd None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:31.861524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff6f12-a449-4e76-935d-d161fed16dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:31.861997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff6f12-a449-4e76-935d-d161fed16dfd None admin] PATH_INFO: `/v3/users/a05cd95cd10645a392f5bcfee57786e7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:31.868688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff6f12-a449-4e76-935d-d161fed16dfd None admin] RBAC: Authorizing `identity:get_user(user_id=a05cd95cd10645a392f5bcfee57786e7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:31.876401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff6f12-a449-4e76-935d-d161fed16dfd None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:31.876401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 103/205] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:31 2022] GET /identity/v3/users/a05cd95cd10645a392f5bcfee57786e7 => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:31.877806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a386f-83d6-4e60-b7c9-ba0e25178563 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:32.017202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a386f-83d6-4e60-b7c9-ba0e25178563 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:32.041448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:32.054521 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494335d-adce-48ce-837b-98b86ce28359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:32.057266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494335d-adce-48ce-837b-98b86ce28359 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:32.057266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494335d-adce-48ce-837b-98b86ce28359 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:32.057266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494335d-adce-48ce-837b-98b86ce28359 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:32.125781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494335d-adce-48ce-837b-98b86ce28359 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:32.127255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494335d-adce-48ce-837b-98b86ce28359 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:32.233982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 103/206] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:32.240894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff6f12-a449-4e76-935d-d161fed16dfd None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:32.348066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:32.384187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff6f12-a449-4e76-935d-d161fed16dfd None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:32.423199 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6aa80-621e-4530-ab85-313c36a44e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:32.423958 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6aa80-621e-4530-ab85-313c36a44e1e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:32.424248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6aa80-621e-4530-ab85-313c36a44e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:32.424528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6aa80-621e-4530-ab85-313c36a44e1e None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:32.425716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6aa80-621e-4530-ab85-313c36a44e1e None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:32.427184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6aa80-621e-4530-ab85-313c36a44e1e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:32.429094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 104/207] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:32 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:32.435011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494335d-adce-48ce-837b-98b86ce28359 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:32.471181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:32.572463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494335d-adce-48ce-837b-98b86ce28359 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:32.615310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d349b-76a1-4224-aff9-7415d5d56171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:32.615969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d349b-76a1-4224-aff9-7415d5d56171 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:32.616255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d349b-76a1-4224-aff9-7415d5d56171 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:32.616531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d349b-76a1-4224-aff9-7415d5d56171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:32.617321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d349b-76a1-4224-aff9-7415d5d56171 None admin] RBAC: Adding query filter params (role.id=324a803dd87d4027b73d871fe46ea228, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b, user.id=a05cd95cd10645a392f5bcfee57786e7) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:32.617635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d349b-76a1-4224-aff9-7415d5d56171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:32.619077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d349b-76a1-4224-aff9-7415d5d56171 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:32.625118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 104/208] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:32 2022] GET /identity/v3/role_assignments?user.id=a05cd95cd10645a392f5bcfee57786e7&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=324a803dd87d4027b73d871fe46ea228 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:32.727845 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f1867-691c-45ca-977d-a014e2b347fb None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:32.728194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f1867-691c-45ca-977d-a014e2b347fb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:32.728413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f1867-691c-45ca-977d-a014e2b347fb None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:32.730080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 105/209] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:32.735703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850565-5b55-4dde-a2b1-db170ca1fee3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:32.736027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850565-5b55-4dde-a2b1-db170ca1fee3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:32.736306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850565-5b55-4dde-a2b1-db170ca1fee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:32.758579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e54b13-6e6f-4ece-9e44-b768743f5df2 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:32.758923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e54b13-6e6f-4ece-9e44-b768743f5df2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:32.759093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e54b13-6e6f-4ece-9e44-b768743f5df2 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:32.760627 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 106/210] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:32.766324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4037-1573-4818-95b3-7c75630a012f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:32.766610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4037-1573-4818-95b3-7c75630a012f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:32.766883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4037-1573-4818-95b3-7c75630a012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:32.775864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e9850565-5b55-4dde-a2b1-db170ca1fee3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:32.777084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e9850565-5b55-4dde-a2b1-db170ca1fee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:32.799362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-848a4037-1573-4818-95b3-7c75630a012f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:32.800605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-848a4037-1573-4818-95b3-7c75630a012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:32.884424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 105/211] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:32.899445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 107/212] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:32.916112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9850565-5b55-4dde-a2b1-db170ca1fee3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:32.927923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a4037-1573-4818-95b3-7c75630a012f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:32.936014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:33.022478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:33.046392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9850565-5b55-4dde-a2b1-db170ca1fee3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:33.066735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a4037-1573-4818-95b3-7c75630a012f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:33.089549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e24f8-0003-4c45-ac24-3cedd1a2a65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:33.090192 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e24f8-0003-4c45-ac24-3cedd1a2a65e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:33.090469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e24f8-0003-4c45-ac24-3cedd1a2a65e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:33.090737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e24f8-0003-4c45-ac24-3cedd1a2a65e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:33.104681 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e24f8-0003-4c45-ac24-3cedd1a2a65e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-254e24f8-0003-4c45-ac24-3cedd1a2a65e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:33.109762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:33.110669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:33.111471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 106/213] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:33.116636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9d3a3-775b-4995-a4bd-b55b5863297f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:33.117302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9d3a3-775b-4995-a4bd-b55b5863297f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:33.117585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9d3a3-775b-4995-a4bd-b55b5863297f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:33.117863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9d3a3-775b-4995-a4bd-b55b5863297f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:33.119053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9d3a3-775b-4995-a4bd-b55b5863297f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:33.120572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9d3a3-775b-4995-a4bd-b55b5863297f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:33.120939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e24f8-0003-4c45-ac24-3cedd1a2a65e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:33.122169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 108/214] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:33.127563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9d3a3-775b-4995-a4bd-b55b5863297f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:33.225264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:33.246299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9d3a3-775b-4995-a4bd-b55b5863297f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:33.251573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e24f8-0003-4c45-ac24-3cedd1a2a65e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:33.287952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15b9c7-900f-47c2-99c9-868ce12a4eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:33.288505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15b9c7-900f-47c2-99c9-868ce12a4eb4 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:33.288791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15b9c7-900f-47c2-99c9-868ce12a4eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:33.289070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15b9c7-900f-47c2-99c9-868ce12a4eb4 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:33.289837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15b9c7-900f-47c2-99c9-868ce12a4eb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:33.291302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15b9c7-900f-47c2-99c9-868ce12a4eb4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:33.293324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bd704-ca6f-4310-8217-a0eab217dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:33.294316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bd704-ca6f-4310-8217-a0eab217dd57 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:33.294812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bd704-ca6f-4310-8217-a0eab217dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:33.295287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bd704-ca6f-4310-8217-a0eab217dd57 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:33.296274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bd704-ca6f-4310-8217-a0eab217dd57 None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:33.296796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bd704-ca6f-4310-8217-a0eab217dd57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:33.298357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bd704-ca6f-4310-8217-a0eab217dd57 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:33.300288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:33.309085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 107/215] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:33.314648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bd704-ca6f-4310-8217-a0eab217dd57 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:33.399211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 109/216] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:04:33 2022] POST /identity/v3/users => generated 297 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:04:33.424774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bd704-ca6f-4310-8217-a0eab217dd57 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:33.467675 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7a5c9-6009-4e4a-a833-9d1168d07d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:33.468770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7a5c9-6009-4e4a-a833-9d1168d07d80 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:33.469332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7a5c9-6009-4e4a-a833-9d1168d07d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:33.469838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7a5c9-6009-4e4a-a833-9d1168d07d80 None admin] PATH_INFO: `/v3/users/9af37e8dc107403988328caa32a19828` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:33.480503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:33.488652 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b7a5c9-6009-4e4a-a833-9d1168d07d80 None admin] RBAC: Authorizing `identity:get_user(user_id=9af37e8dc107403988328caa32a19828)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:33.491311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b7a5c9-6009-4e4a-a833-9d1168d07d80 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:33.493562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 108/217] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:33 2022] GET /identity/v3/users/9af37e8dc107403988328caa32a19828 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:33.499607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15b9c7-900f-47c2-99c9-868ce12a4eb4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:33.610876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15b9c7-900f-47c2-99c9-868ce12a4eb4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:33.647987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c9f86-b544-4ba4-b2a8-ced6756bcefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:33.648619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c9f86-b544-4ba4-b2a8-ced6756bcefb None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:33.648905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c9f86-b544-4ba4-b2a8-ced6756bcefb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:33.649182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c9f86-b544-4ba4-b2a8-ced6756bcefb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:33.715842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:33.726211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c9f86-b544-4ba4-b2a8-ced6756bcefb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:33.727677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c9f86-b544-4ba4-b2a8-ced6756bcefb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:33.841212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 110/218] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:33.846376 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7a5c9-6009-4e4a-a833-9d1168d07d80 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:33.958718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7a5c9-6009-4e4a-a833-9d1168d07d80 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:34.004655 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82758065-3bf8-4b0f-9772-a4624915fcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:34.005823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82758065-3bf8-4b0f-9772-a4624915fcfa None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:34.006308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82758065-3bf8-4b0f-9772-a4624915fcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:34.006796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82758065-3bf8-4b0f-9772-a4624915fcfa None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:34.008239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82758065-3bf8-4b0f-9772-a4624915fcfa None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:34.009908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82758065-3bf8-4b0f-9772-a4624915fcfa None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:34.011825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 109/219] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:33 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:34.018037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c9f86-b544-4ba4-b2a8-ced6756bcefb None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:34.067511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d47a63-c804-4710-96ed-6794896e974d None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:34.067855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d47a63-c804-4710-96ed-6794896e974d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:34.068113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d47a63-c804-4710-96ed-6794896e974d None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:34.070316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 110/220] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:34.075119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53b41d-3b23-4c89-80ed-11902196cfed None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:34.075662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53b41d-3b23-4c89-80ed-11902196cfed None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:34.075999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53b41d-3b23-4c89-80ed-11902196cfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:34.094749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8b53b41d-3b23-4c89-80ed-11902196cfed None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:34.095946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8b53b41d-3b23-4c89-80ed-11902196cfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:34.157162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:34.173714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c9f86-b544-4ba4-b2a8-ced6756bcefb None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:34.204597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:34.214705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 111/221] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:34.242263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53b41d-3b23-4c89-80ed-11902196cfed None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:34.248486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be9790-b364-4229-af64-c7c8c7b58974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:34.249127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be9790-b364-4229-af64-c7c8c7b58974 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:34.249410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be9790-b364-4229-af64-c7c8c7b58974 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:34.249682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be9790-b364-4229-af64-c7c8c7b58974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:34.250539 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be9790-b364-4229-af64-c7c8c7b58974 None admin] RBAC: Adding query filter params (role.id=512a743b8ca748f8b793681748be7906, scope.project.id=3059810e56d143ab91728e2220010d53, user.id=9af37e8dc107403988328caa32a19828) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:34.250852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be9790-b364-4229-af64-c7c8c7b58974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:34.252330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be9790-b364-4229-af64-c7c8c7b58974 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:34.269717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 111/222] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:34 2022] GET /identity/v3/role_assignments?user.id=9af37e8dc107403988328caa32a19828&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=512a743b8ca748f8b793681748be7906 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:34.288380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3389fe1-b43d-4fd6-b285-e8e60446365d None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:34.288666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3389fe1-b43d-4fd6-b285-e8e60446365d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:34.288981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3389fe1-b43d-4fd6-b285-e8e60446365d None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:34.290389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 112/223] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:34.296943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d967c4-35d2-458c-99a2-3b46187caf13 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:34.297231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d967c4-35d2-458c-99a2-3b46187caf13 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:34.297509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d967c4-35d2-458c-99a2-3b46187caf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:34.329104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-55d967c4-35d2-458c-99a2-3b46187caf13 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:34.330311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-55d967c4-35d2-458c-99a2-3b46187caf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:34.393764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53b41d-3b23-4c89-80ed-11902196cfed None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:34.453907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 113/224] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:34.461348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279c213-12a1-4733-ab68-88ff42ecd9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:34.462084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c213-12a1-4733-ab68-88ff42ecd9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:34.462436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c213-12a1-4733-ab68-88ff42ecd9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:34.462786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c213-12a1-4733-ab68-88ff42ecd9e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:34.474947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279c213-12a1-4733-ab68-88ff42ecd9e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:34.476705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d967c4-35d2-458c-99a2-3b46187caf13 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9279c213-12a1-4733-ab68-88ff42ecd9e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:34.481676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:34.482553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:34.483372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 112/225] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:34.491924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279c213-12a1-4733-ab68-88ff42ecd9e3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:34.606304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:34.649171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:34.671553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d967c4-35d2-458c-99a2-3b46187caf13 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:34.689936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279c213-12a1-4733-ab68-88ff42ecd9e3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:34.726090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe370fe0-8b9c-430d-abab-26d3859da42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:34.727066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370fe0-8b9c-430d-abab-26d3859da42c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:34.727551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370fe0-8b9c-430d-abab-26d3859da42c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:34.728053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370fe0-8b9c-430d-abab-26d3859da42c None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:34.728966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe370fe0-8b9c-430d-abab-26d3859da42c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:34.729849 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe370fe0-8b9c-430d-abab-26d3859da42c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:34.732196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe370fe0-8b9c-430d-abab-26d3859da42c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:34.740749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 113/226] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:34.746486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe370fe0-8b9c-430d-abab-26d3859da42c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:34.753370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9795c-a2a8-490f-b4d5-4a1a4de59c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:34.754042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9795c-a2a8-490f-b4d5-4a1a4de59c29 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:34.754325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9795c-a2a8-490f-b4d5-4a1a4de59c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:34.754649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9795c-a2a8-490f-b4d5-4a1a4de59c29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:34.780468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9795c-a2a8-490f-b4d5-4a1a4de59c29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b6a9795c-a2a8-490f-b4d5-4a1a4de59c29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:34.785523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:34.786394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:34.788370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 114/227] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:34.793694 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9795c-a2a8-490f-b4d5-4a1a4de59c29 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:34.893939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe370fe0-8b9c-430d-abab-26d3859da42c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:34.918352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:34.936415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9795c-a2a8-490f-b4d5-4a1a4de59c29 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:34.954412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b85c9-d135-4322-ba08-53275d148b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:34.955081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b85c9-d135-4322-ba08-53275d148b43 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:34.955367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b85c9-d135-4322-ba08-53275d148b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:34.955643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b85c9-d135-4322-ba08-53275d148b43 None admin] PATH_INFO: `/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:34.958291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b85c9-d135-4322-ba08-53275d148b43 None admin] RBAC: Authorizing `identity:get_user(user_id=e558a5118d9149a78c5bc1a4f4dbb7b7)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:34.960374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b85c9-d135-4322-ba08-53275d148b43 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:34.962041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 114/228] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:34 2022] GET /identity/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:34.967902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b85c9-d135-4322-ba08-53275d148b43 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:35.006440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a252b-109e-4056-8cc7-adf4630689cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:35.007094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a252b-109e-4056-8cc7-adf4630689cc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:35.007378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a252b-109e-4056-8cc7-adf4630689cc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:35.007655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a252b-109e-4056-8cc7-adf4630689cc None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:35.008438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a252b-109e-4056-8cc7-adf4630689cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:35.011108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a252b-109e-4056-8cc7-adf4630689cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:35.012655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a252b-109e-4056-8cc7-adf4630689cc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:35.028931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 115/229] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:35.034773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a252b-109e-4056-8cc7-adf4630689cc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:35.049800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:35.163284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a252b-109e-4056-8cc7-adf4630689cc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:35.173926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b85c9-d135-4322-ba08-53275d148b43 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:35.199299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f244cc4-1c84-481b-b1cb-bcd98228158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:35.199998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f244cc4-1c84-481b-b1cb-bcd98228158b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:35.200282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f244cc4-1c84-481b-b1cb-bcd98228158b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:35.200815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f244cc4-1c84-481b-b1cb-bcd98228158b None admin] PATH_INFO: `/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:35.202274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f244cc4-1c84-481b-b1cb-bcd98228158b None admin] RBAC: Authorizing `identity:get_user(user_id=e558a5118d9149a78c5bc1a4f4dbb7b7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:35.203925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f244cc4-1c84-481b-b1cb-bcd98228158b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:35.206218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 116/230] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:35 2022] GET /identity/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:35.212463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f244cc4-1c84-481b-b1cb-bcd98228158b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:35.250699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49851b8-3049-4f93-b14b-b8802e5729ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:35.251347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49851b8-3049-4f93-b14b-b8802e5729ce None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:35.251677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49851b8-3049-4f93-b14b-b8802e5729ce None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:35.251987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49851b8-3049-4f93-b14b-b8802e5729ce None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:35.253206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49851b8-3049-4f93-b14b-b8802e5729ce None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:35.254665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49851b8-3049-4f93-b14b-b8802e5729ce None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:35.257292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 115/231] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:34 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:35.268741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49851b8-3049-4f93-b14b-b8802e5729ce None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:35.321435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:35.349622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:35.368179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f244cc4-1c84-481b-b1cb-bcd98228158b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:35.379836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49851b8-3049-4f93-b14b-b8802e5729ce None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:35.416321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079beab-f486-4e72-ba3a-2368f504218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:35.417100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079beab-f486-4e72-ba3a-2368f504218a None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:35.417460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079beab-f486-4e72-ba3a-2368f504218a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:35.417814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079beab-f486-4e72-ba3a-2368f504218a None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53/users/e558a5118d9149a78c5bc1a4f4dbb7b7/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:35.419900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079beab-f486-4e72-ba3a-2368f504218a None admin] RBAC: Authorizing `identity:create_grant(project_id=3059810e56d143ab91728e2220010d53, user_id=e558a5118d9149a78c5bc1a4f4dbb7b7, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:35.421705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079beab-f486-4e72-ba3a-2368f504218a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:35.441556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d0e10-68dc-467b-a13b-c8979294ccc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:35.442144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d0e10-68dc-467b-a13b-c8979294ccc0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:35.442476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d0e10-68dc-467b-a13b-c8979294ccc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:35.442763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d0e10-68dc-467b-a13b-c8979294ccc0 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:35.444535 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d0e10-68dc-467b-a13b-c8979294ccc0 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:35.446130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d0e10-68dc-467b-a13b-c8979294ccc0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:35.448187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 117/232] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:35 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:35.454215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d0e10-68dc-467b-a13b-c8979294ccc0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:35.498450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 116/233] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:04:35 2022] PUT /identity/v3/projects/3059810e56d143ab91728e2220010d53/users/e558a5118d9149a78c5bc1a4f4dbb7b7/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:35.581080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d0e10-68dc-467b-a13b-c8979294ccc0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:35.625308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b7075-92d9-4b38-8bff-0f175b909992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:35.625999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b7075-92d9-4b38-8bff-0f175b909992 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:35.626283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b7075-92d9-4b38-8bff-0f175b909992 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:35.626559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b7075-92d9-4b38-8bff-0f175b909992 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53/users/e558a5118d9149a78c5bc1a4f4dbb7b7/roles/b5b241afd53c456e9a9fa3033eea74ad` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:35.628991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b7075-92d9-4b38-8bff-0f175b909992 None admin] RBAC: Authorizing `identity:create_grant(project_id=3059810e56d143ab91728e2220010d53, user_id=e558a5118d9149a78c5bc1a4f4dbb7b7, role_id=b5b241afd53c456e9a9fa3033eea74ad)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:35.631179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b7075-92d9-4b38-8bff-0f175b909992 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:35.706379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 118/234] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:04:35 2022] PUT /identity/v3/projects/3059810e56d143ab91728e2220010d53/users/e558a5118d9149a78c5bc1a4f4dbb7b7/roles/b5b241afd53c456e9a9fa3033eea74ad => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:04:35.938744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75d631-09c7-4b7c-b398-69b396b69bdc None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:35.939105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75d631-09c7-4b7c-b398-69b396b69bdc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:35.939302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75d631-09c7-4b7c-b398-69b396b69bdc None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:35.940871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 117/235] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:35.947051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a634107-8136-46d1-8f4a-842db6ed512a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:35.947346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a634107-8136-46d1-8f4a-842db6ed512a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:35.947626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a634107-8136-46d1-8f4a-842db6ed512a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:35.956853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:35.969571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4a634107-8136-46d1-8f4a-842db6ed512a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:35.970779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4a634107-8136-46d1-8f4a-842db6ed512a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:36.043531 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 119/236] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:36.067328 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75d631-09c7-4b7c-b398-69b396b69bdc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:36.093127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccb27f-f9b0-4f4d-a1a3-1185b1735967 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:36.093443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccb27f-f9b0-4f4d-a1a3-1185b1735967 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:36.093697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccb27f-f9b0-4f4d-a1a3-1185b1735967 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:36.095075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 120/237] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:36.100651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ad45-b3dc-4e90-a7f2-aa514aed8b54 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:36.100939 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ad45-b3dc-4e90-a7f2-aa514aed8b54 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:36.101220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ad45-b3dc-4e90-a7f2-aa514aed8b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:36.138393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-97c5ad45-b3dc-4e90-a7f2-aa514aed8b54 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:36.140832 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-97c5ad45-b3dc-4e90-a7f2-aa514aed8b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:36.222854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 121/238] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:36.235327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75d631-09c7-4b7c-b398-69b396b69bdc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:36.256785 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5ad45-b3dc-4e90-a7f2-aa514aed8b54 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:36.267665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:36.297511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122d129-c26c-410e-80d2-cf5faf2734e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:36.298169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122d129-c26c-410e-80d2-cf5faf2734e3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:36.298450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122d129-c26c-410e-80d2-cf5faf2734e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:36.299001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122d129-c26c-410e-80d2-cf5faf2734e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:36.301717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122d129-c26c-410e-80d2-cf5faf2734e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:36.305603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122d129-c26c-410e-80d2-cf5faf2734e3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:36.307294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 118/239] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:36.328320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122d129-c26c-410e-80d2-cf5faf2734e3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:36.383175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:36.474946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122d129-c26c-410e-80d2-cf5faf2734e3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:36.485724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5ad45-b3dc-4e90-a7f2-aa514aed8b54 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:36.521261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ae06d-9dc6-4cb4-a031-9398d9fbdae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:36.522714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ae06d-9dc6-4cb4-a031-9398d9fbdae6 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:36.524113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ae06d-9dc6-4cb4-a031-9398d9fbdae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:36.524570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ae06d-9dc6-4cb4-a031-9398d9fbdae6 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:36.525806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ae06d-9dc6-4cb4-a031-9398d9fbdae6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:36.528697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ae06d-9dc6-4cb4-a031-9398d9fbdae6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:36.552735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cebec-94b0-418b-bf0e-67f546a85f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:36.553718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cebec-94b0-418b-bf0e-67f546a85f82 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:36.554246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cebec-94b0-418b-bf0e-67f546a85f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:36.554662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cebec-94b0-418b-bf0e-67f546a85f82 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:36.565795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cebec-94b0-418b-bf0e-67f546a85f82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-753cebec-94b0-418b-bf0e-67f546a85f82 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:36.571579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:36.572688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:36.573705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:36.575999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 122/240] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:36.585307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cebec-94b0-418b-bf0e-67f546a85f82 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:36.618748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 119/241] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:04:36 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:36.699708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cebec-94b0-418b-bf0e-67f546a85f82 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:36.727134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:36.743408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fad736-fd24-4c39-9eb0-5b1cdb2d8981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:36.744088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fad736-fd24-4c39-9eb0-5b1cdb2d8981 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:36.744374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fad736-fd24-4c39-9eb0-5b1cdb2d8981 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:36.744650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fad736-fd24-4c39-9eb0-5b1cdb2d8981 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:36.745398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fad736-fd24-4c39-9eb0-5b1cdb2d8981 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:36.745707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fad736-fd24-4c39-9eb0-5b1cdb2d8981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:36.747104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fad736-fd24-4c39-9eb0-5b1cdb2d8981 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:36.757055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 123/242] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:36.763023 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c197fe-e017-41b1-96b0-9286f4614aec None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:36.763306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c197fe-e017-41b1-96b0-9286f4614aec None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:36.763592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c197fe-e017-41b1-96b0-9286f4614aec None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:36.763869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fad736-fd24-4c39-9eb0-5b1cdb2d8981 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:36.765041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 120/243] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:36.770502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8902c4d-14ad-47f0-a307-a257d40e98c5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:36.770792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8902c4d-14ad-47f0-a307-a257d40e98c5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:36.771079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8902c4d-14ad-47f0-a307-a257d40e98c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:36.817051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f8902c4d-14ad-47f0-a307-a257d40e98c5 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:36.818960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f8902c4d-14ad-47f0-a307-a257d40e98c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:36.872179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:36.880599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fad736-fd24-4c39-9eb0-5b1cdb2d8981 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:36.917640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 121/244] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:36.922085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddec30-4f10-4f3b-aae7-79b4dd364012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:36.922760 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddec30-4f10-4f3b-aae7-79b4dd364012 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:36.923045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddec30-4f10-4f3b-aae7-79b4dd364012 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:36.923323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddec30-4f10-4f3b-aae7-79b4dd364012 None admin] PATH_INFO: `/v3/users/973fa099ee414f60b501373c88a8268f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:36.941586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcddec30-4f10-4f3b-aae7-79b4dd364012 None admin] RBAC: Authorizing `identity:get_user(user_id=973fa099ee414f60b501373c88a8268f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:36.943923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcddec30-4f10-4f3b-aae7-79b4dd364012 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:36.945178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8902c4d-14ad-47f0-a307-a257d40e98c5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:36.945937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 124/245] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:36 2022] GET /identity/v3/users/973fa099ee414f60b501373c88a8268f => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:36.951767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddec30-4f10-4f3b-aae7-79b4dd364012 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:36.999413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:37.053339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8902c4d-14ad-47f0-a307-a257d40e98c5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:37.057039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddec30-4f10-4f3b-aae7-79b4dd364012 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:37.093939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b45fe-0f58-460c-8103-0e33a033c17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:37.095223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f211658-8c92-4481-be22-d09d6ad6493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:37.095479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b45fe-0f58-460c-8103-0e33a033c17f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:37.095876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211658-8c92-4481-be22-d09d6ad6493e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:37.096157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211658-8c92-4481-be22-d09d6ad6493e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:37.096429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211658-8c92-4481-be22-d09d6ad6493e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:37.096528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b45fe-0f58-460c-8103-0e33a033c17f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:37.096996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b45fe-0f58-460c-8103-0e33a033c17f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:37.111358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b45fe-0f58-460c-8103-0e33a033c17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bf0b45fe-0f58-460c-8103-0e33a033c17f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:37.117409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:37.118281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:37.119138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:37.121067 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 122/246] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:37.127756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b45fe-0f58-460c-8103-0e33a033c17f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:37.167888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:37.197714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f211658-8c92-4481-be22-d09d6ad6493e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:37.199194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f211658-8c92-4481-be22-d09d6ad6493e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:37.255368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:37.272614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b45fe-0f58-460c-8103-0e33a033c17f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:37.316494 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7201b32-1a43-49b5-9c04-9f242f06ed85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:37.317113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7201b32-1a43-49b5-9c04-9f242f06ed85 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:37.317388 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7201b32-1a43-49b5-9c04-9f242f06ed85 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:37.317659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7201b32-1a43-49b5-9c04-9f242f06ed85 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:37.318409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7201b32-1a43-49b5-9c04-9f242f06ed85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:37.318771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7201b32-1a43-49b5-9c04-9f242f06ed85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:37.321027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7201b32-1a43-49b5-9c04-9f242f06ed85 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:37.321891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 125/247] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:37.327429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f211658-8c92-4481-be22-d09d6ad6493e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:37.341207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 123/248] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:37.349308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7201b32-1a43-49b5-9c04-9f242f06ed85 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:37.440902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f211658-8c92-4481-be22-d09d6ad6493e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:37.458748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:37.475334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7201b32-1a43-49b5-9c04-9f242f06ed85 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:37.485660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179746ef-e3f1-4a9a-907a-0bf55196c964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:37.486319 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179746ef-e3f1-4a9a-907a-0bf55196c964 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:37.486605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179746ef-e3f1-4a9a-907a-0bf55196c964 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:37.486885 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179746ef-e3f1-4a9a-907a-0bf55196c964 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:37.488104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179746ef-e3f1-4a9a-907a-0bf55196c964 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:37.489644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179746ef-e3f1-4a9a-907a-0bf55196c964 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:37.492681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 126/249] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:37 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:37.500673 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179746ef-e3f1-4a9a-907a-0bf55196c964 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:37.520559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:37.527870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732adbe2-1c2a-4e1c-a494-157500f65807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:37.528820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732adbe2-1c2a-4e1c-a494-157500f65807 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:37.530183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732adbe2-1c2a-4e1c-a494-157500f65807 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:37.531333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732adbe2-1c2a-4e1c-a494-157500f65807 None admin] PATH_INFO: `/v3/users/9af37e8dc107403988328caa32a19828` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:37.532909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732adbe2-1c2a-4e1c-a494-157500f65807 None admin] RBAC: Authorizing `identity:get_user(user_id=9af37e8dc107403988328caa32a19828)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:37.536568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732adbe2-1c2a-4e1c-a494-157500f65807 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:37.540807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 124/250] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:37 2022] GET /identity/v3/users/9af37e8dc107403988328caa32a19828 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:37.547265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732adbe2-1c2a-4e1c-a494-157500f65807 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:37.679453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732adbe2-1c2a-4e1c-a494-157500f65807 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:37.727759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0498e-f72d-4fb5-9a36-ad9b5c4b774f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:37.728445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0498e-f72d-4fb5-9a36-ad9b5c4b774f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:37.728721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0498e-f72d-4fb5-9a36-ad9b5c4b774f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:37.729000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0498e-f72d-4fb5-9a36-ad9b5c4b774f None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:37.731326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0498e-f72d-4fb5-9a36-ad9b5c4b774f None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:37.732866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0498e-f72d-4fb5-9a36-ad9b5c4b774f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:37.735687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 125/251] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:37 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:37.741155 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0498e-f72d-4fb5-9a36-ad9b5c4b774f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:37.761139 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179746ef-e3f1-4a9a-907a-0bf55196c964 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:37.798707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:37.828743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818432d5-e701-43af-9d23-0a76c30fefb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:37.829740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818432d5-e701-43af-9d23-0a76c30fefb3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:37.830054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818432d5-e701-43af-9d23-0a76c30fefb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:37.830335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818432d5-e701-43af-9d23-0a76c30fefb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:37.831191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818432d5-e701-43af-9d23-0a76c30fefb3 None admin] RBAC: Adding query filter params (role.id=324a803dd87d4027b73d871fe46ea228, scope.project.id=3059810e56d143ab91728e2220010d53, user.id=973fa099ee414f60b501373c88a8268f) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:37.831512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818432d5-e701-43af-9d23-0a76c30fefb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:37.833013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818432d5-e701-43af-9d23-0a76c30fefb3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:37.839746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 127/252] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:37 2022] GET /identity/v3/role_assignments?user.id=973fa099ee414f60b501373c88a8268f&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=324a803dd87d4027b73d871fe46ea228 => generated 259 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:37.909744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0498e-f72d-4fb5-9a36-ad9b5c4b774f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:37.963540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefcd84-e188-4d40-8005-fa2098d1f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:37.964265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefcd84-e188-4d40-8005-fa2098d1f60a None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:37.964556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefcd84-e188-4d40-8005-fa2098d1f60a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:37.964834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefcd84-e188-4d40-8005-fa2098d1f60a None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53/users/9af37e8dc107403988328caa32a19828/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:37.970496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefcd84-e188-4d40-8005-fa2098d1f60a None admin] RBAC: Authorizing `identity:create_grant(project_id=3059810e56d143ab91728e2220010d53, user_id=9af37e8dc107403988328caa32a19828, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:37.972014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefcd84-e188-4d40-8005-fa2098d1f60a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:38.047956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 126/253] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:04:37 2022] PUT /identity/v3/projects/3059810e56d143ab91728e2220010d53/users/9af37e8dc107403988328caa32a19828/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:38.183283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f0051-2d04-4120-b7c5-5837484a30e1 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:38.183920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f0051-2d04-4120-b7c5-5837484a30e1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:38.183920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f0051-2d04-4120-b7c5-5837484a30e1 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:38.185405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 128/254] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:38.194913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf323c-c4f6-48b9-b05f-aeed58dfb607 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:38.195531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf323c-c4f6-48b9-b05f-aeed58dfb607 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:38.195531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf323c-c4f6-48b9-b05f-aeed58dfb607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:38.225099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e0bf323c-c4f6-48b9-b05f-aeed58dfb607 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:38.226369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e0bf323c-c4f6-48b9-b05f-aeed58dfb607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:38.321866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 127/255] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:38.359725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f0051-2d04-4120-b7c5-5837484a30e1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:38.396342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265c68e-c5a1-434a-8c26-86c4069d9420 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:38.396663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265c68e-c5a1-434a-8c26-86c4069d9420 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:38.396908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265c68e-c5a1-434a-8c26-86c4069d9420 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:38.398571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 128/256] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:38.407808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9654d-0ca6-4986-b7b0-f0b89b027836 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:38.407808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9654d-0ca6-4986-b7b0-f0b89b027836 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:38.407808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9654d-0ca6-4986-b7b0-f0b89b027836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:38.426370 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-40d9654d-0ca6-4986-b7b0-f0b89b027836 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:38.428503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-40d9654d-0ca6-4986-b7b0-f0b89b027836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:38.444112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:38.491879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:38.532341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f0051-2d04-4120-b7c5-5837484a30e1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:38.534417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 129/257] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:38.563318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9654d-0ca6-4986-b7b0-f0b89b027836 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:38.601437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a6232-a2d2-4d05-888c-de17a2d42de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:38.602114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a6232-a2d2-4d05-888c-de17a2d42de4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:38.602453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a6232-a2d2-4d05-888c-de17a2d42de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:38.602734 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a6232-a2d2-4d05-888c-de17a2d42de4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:38.621854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a6232-a2d2-4d05-888c-de17a2d42de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-a52a6232-a2d2-4d05-888c-de17a2d42de4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:38.634042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:38.634993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:38.635815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:38.640260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 129/258] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:38.644206 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a6232-a2d2-4d05-888c-de17a2d42de4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:38.819710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9654d-0ca6-4986-b7b0-f0b89b027836 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:38.876984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb90385-76e4-4fc2-8bf0-85b62d10a7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:38.877712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb90385-76e4-4fc2-8bf0-85b62d10a7dd None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:38.878004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb90385-76e4-4fc2-8bf0-85b62d10a7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:38.878285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb90385-76e4-4fc2-8bf0-85b62d10a7dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:38.899043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb90385-76e4-4fc2-8bf0-85b62d10a7dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-beb90385-76e4-4fc2-8bf0-85b62d10a7dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:38.908748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:38.910281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:38.911092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:38.912732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 130/259] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:38.918789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb90385-76e4-4fc2-8bf0-85b62d10a7dd None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:38.925266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:38.929709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a6232-a2d2-4d05-888c-de17a2d42de4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:39.014581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95ffb-b4b8-487d-ae23-3bf47b0af596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:39.015505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95ffb-b4b8-487d-ae23-3bf47b0af596 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:39.015832 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95ffb-b4b8-487d-ae23-3bf47b0af596 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:39.016090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95ffb-b4b8-487d-ae23-3bf47b0af596 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:39.017235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a95ffb-b4b8-487d-ae23-3bf47b0af596 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:39.017649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a95ffb-b4b8-487d-ae23-3bf47b0af596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:39.019111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a95ffb-b4b8-487d-ae23-3bf47b0af596 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:39.033946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:39.038603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 130/260] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:39.039937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95ffb-b4b8-487d-ae23-3bf47b0af596 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:39.186279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95ffb-b4b8-487d-ae23-3bf47b0af596 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:39.241061 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb832953-a9da-4bc2-8225-90170346404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:39.241729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb832953-a9da-4bc2-8225-90170346404e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:39.242070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb832953-a9da-4bc2-8225-90170346404e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:39.242356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb832953-a9da-4bc2-8225-90170346404e None admin] PATH_INFO: `/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:39.244493 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb832953-a9da-4bc2-8225-90170346404e None admin] RBAC: Authorizing `identity:get_user(user_id=e558a5118d9149a78c5bc1a4f4dbb7b7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:39.246517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb832953-a9da-4bc2-8225-90170346404e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:39.248608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 131/261] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:39 2022] GET /identity/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:39.255157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb832953-a9da-4bc2-8225-90170346404e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:39.293785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb90385-76e4-4fc2-8bf0-85b62d10a7dd None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:39.381479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec97a2-0705-4236-a380-40d98c4c3fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:39.382108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec97a2-0705-4236-a380-40d98c4c3fde None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:39.382392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec97a2-0705-4236-a380-40d98c4c3fde None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:39.382667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec97a2-0705-4236-a380-40d98c4c3fde None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:39.383397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec97a2-0705-4236-a380-40d98c4c3fde None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:39.383705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec97a2-0705-4236-a380-40d98c4c3fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:39.385111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec97a2-0705-4236-a380-40d98c4c3fde None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:39.392197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:39.404161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 131/262] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:39.409767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec97a2-0705-4236-a380-40d98c4c3fde None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:39.464087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb832953-a9da-4bc2-8225-90170346404e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:39.525425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69e86d-f7d3-48fe-9c3c-6002defc2eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:39.526077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69e86d-f7d3-48fe-9c3c-6002defc2eca None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:39.526367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69e86d-f7d3-48fe-9c3c-6002defc2eca None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:39.526651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69e86d-f7d3-48fe-9c3c-6002defc2eca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:39.562926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:39.660466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69e86d-f7d3-48fe-9c3c-6002defc2eca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:39.662501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69e86d-f7d3-48fe-9c3c-6002defc2eca None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:39.665773 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec97a2-0705-4236-a380-40d98c4c3fde None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:39.728053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:39.758524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23635ad2-6ff6-4834-aa14-b318a8001fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:39.759186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23635ad2-6ff6-4834-aa14-b318a8001fdf None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:39.759478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23635ad2-6ff6-4834-aa14-b318a8001fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:39.759813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23635ad2-6ff6-4834-aa14-b318a8001fdf None admin] PATH_INFO: `/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:39.760972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23635ad2-6ff6-4834-aa14-b318a8001fdf None admin] RBAC: Authorizing `identity:get_user(user_id=e558a5118d9149a78c5bc1a4f4dbb7b7)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:39.762454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23635ad2-6ff6-4834-aa14-b318a8001fdf None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:39.772780 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 132/263] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:39 2022] GET /identity/v3/users/e558a5118d9149a78c5bc1a4f4dbb7b7 => generated 283 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:39.780729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23635ad2-6ff6-4834-aa14-b318a8001fdf None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:39.784050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 132/264] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:39.790879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69e86d-f7d3-48fe-9c3c-6002defc2eca None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:39.907483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69e86d-f7d3-48fe-9c3c-6002defc2eca None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:39.997967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23635ad2-6ff6-4834-aa14-b318a8001fdf None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:40.001407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7973d2-1b07-4c75-b094-abc21c006e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:40.002071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7973d2-1b07-4c75-b094-abc21c006e96 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.002355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7973d2-1b07-4c75-b094-abc21c006e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.002639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7973d2-1b07-4c75-b094-abc21c006e96 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.011806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7973d2-1b07-4c75-b094-abc21c006e96 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:40.011806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7973d2-1b07-4c75-b094-abc21c006e96 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:40.012185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 133/265] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:39 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:40.018695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7973d2-1b07-4c75-b094-abc21c006e96 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:40.057651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06e0b7-d42b-466e-840b-a71575cebf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:40.058627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06e0b7-d42b-466e-840b-a71575cebf7e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.059115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06e0b7-d42b-466e-840b-a71575cebf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.059650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06e0b7-d42b-466e-840b-a71575cebf7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.066278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:40.071112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:40.143808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7973d2-1b07-4c75-b094-abc21c006e96 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:40.178839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114806f-2844-467e-aa4a-31624406676e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:40.179469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114806f-2844-467e-aa4a-31624406676e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.179753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114806f-2844-467e-aa4a-31624406676e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.180063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114806f-2844-467e-aa4a-31624406676e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.180866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114806f-2844-467e-aa4a-31624406676e None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=3059810e56d143ab91728e2220010d53, user.id=e558a5118d9149a78c5bc1a4f4dbb7b7) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:40.181179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114806f-2844-467e-aa4a-31624406676e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:40.182631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114806f-2844-467e-aa4a-31624406676e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:40.194802 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 134/266] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:40 2022] GET /identity/v3/role_assignments?user.id=e558a5118d9149a78c5bc1a4f4dbb7b7&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:40.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06e0b7-d42b-466e-840b-a71575cebf7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:40.202266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06e0b7-d42b-466e-840b-a71575cebf7e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:40.232969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a8c42-928f-4289-be35-1d71bb45360e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.233276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a8c42-928f-4289-be35-1d71bb45360e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.233526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a8c42-928f-4289-be35-1d71bb45360e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.235149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 135/267] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:40.244092 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373514-19bf-4cef-bb24-738f35467fe3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.244522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373514-19bf-4cef-bb24-738f35467fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.245127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c373514-19bf-4cef-bb24-738f35467fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.273138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3c373514-19bf-4cef-bb24-738f35467fe3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:40.274305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3c373514-19bf-4cef-bb24-738f35467fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:40.398039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 133/268] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:40.399160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13bb89-b7fe-4fa7-9e75-9271549f6b3e None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.399455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13bb89-b7fe-4fa7-9e75-9271549f6b3e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.402198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13bb89-b7fe-4fa7-9e75-9271549f6b3e None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.402198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 134/269] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:40.408402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 136/270] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:40.409429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13bb89-b7fe-4fa7-9e75-9271549f6b3e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:40.410901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac5649-65d1-4a7f-acb0-f272dbf4c64c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.411364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac5649-65d1-4a7f-acb0-f272dbf4c64c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.411982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac5649-65d1-4a7f-acb0-f272dbf4c64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.438751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9bac5649-65d1-4a7f-acb0-f272dbf4c64c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:40.440813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9bac5649-65d1-4a7f-acb0-f272dbf4c64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:40.471075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:40.477505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:40.537233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 137/271] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:40.538140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac5649-65d1-4a7f-acb0-f272dbf4c64c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:40.571440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13bb89-b7fe-4fa7-9e75-9271549f6b3e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:40.635202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcc494-61e1-45ac-a017-f92e032b304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:40.636228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcc494-61e1-45ac-a017-f92e032b304f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.636858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcc494-61e1-45ac-a017-f92e032b304f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.637282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcc494-61e1-45ac-a017-f92e032b304f None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.638663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbcc494-61e1-45ac-a017-f92e032b304f None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:40.641481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbcc494-61e1-45ac-a017-f92e032b304f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:40.643362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 135/272] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:40 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:40.644613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcc494-61e1-45ac-a017-f92e032b304f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:40.677116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac5649-65d1-4a7f-acb0-f272dbf4c64c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:40.715664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4e9f8-a9b7-49b2-8964-3eff2ec66142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:40.716574 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4e9f8-a9b7-49b2-8964-3eff2ec66142 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.716858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4e9f8-a9b7-49b2-8964-3eff2ec66142 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.717134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4e9f8-a9b7-49b2-8964-3eff2ec66142 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.718279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4e9f8-a9b7-49b2-8964-3eff2ec66142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:40.719753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4e9f8-a9b7-49b2-8964-3eff2ec66142 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:40.721551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 138/273] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:40.722541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fae318-83f2-4ec8-9952-818df22e5bbe None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.722827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fae318-83f2-4ec8-9952-818df22e5bbe None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.723102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fae318-83f2-4ec8-9952-818df22e5bbe None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.724506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 139/274] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:40.728960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fae318-83f2-4ec8-9952-818df22e5bbe None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:40.750384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcc494-61e1-45ac-a017-f92e032b304f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:40.751934 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:40.761195 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:40.785690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6b012-7df8-403c-86de-c72e0e1039ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:40.786331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6b012-7df8-403c-86de-c72e0e1039ea None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.786608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6b012-7df8-403c-86de-c72e0e1039ea None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.786879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6b012-7df8-403c-86de-c72e0e1039ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.798076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6b012-7df8-403c-86de-c72e0e1039ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-c7a6b012-7df8-403c-86de-c72e0e1039ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:40.803421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:40.804520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 136/275] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:40.805512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6b012-7df8-403c-86de-c72e0e1039ea None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:40.854714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fae318-83f2-4ec8-9952-818df22e5bbe None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:40.889294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670e658-1f03-426c-93db-32221616207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:40.889912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670e658-1f03-426c-93db-32221616207c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.890191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670e658-1f03-426c-93db-32221616207c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.890460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670e658-1f03-426c-93db-32221616207c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.891236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670e658-1f03-426c-93db-32221616207c None admin] RBAC: Adding query filter params (role.id=b5b241afd53c456e9a9fa3033eea74ad, scope.project.id=3059810e56d143ab91728e2220010d53, user.id=e558a5118d9149a78c5bc1a4f4dbb7b7) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:40.891542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670e658-1f03-426c-93db-32221616207c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:40.893026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670e658-1f03-426c-93db-32221616207c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:40.899008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 140/276] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:40 2022] GET /identity/v3/role_assignments?user.id=e558a5118d9149a78c5bc1a4f4dbb7b7&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=b5b241afd53c456e9a9fa3033eea74ad => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:40.901051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8823057-b960-4216-9545-9e037ed18981 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:40.901477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8823057-b960-4216-9545-9e037ed18981 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:40.901884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8823057-b960-4216-9545-9e037ed18981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:40.945393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b8823057-b960-4216-9545-9e037ed18981 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:40.947770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b8823057-b960-4216-9545-9e037ed18981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:40.965807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6b012-7df8-403c-86de-c72e0e1039ea None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:41.003375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6fdd7-3410-4823-b8e0-68f8a2ea1b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:41.004658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6fdd7-3410-4823-b8e0-68f8a2ea1b7c None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.005172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6fdd7-3410-4823-b8e0-68f8a2ea1b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.005734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6fdd7-3410-4823-b8e0-68f8a2ea1b7c None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.006560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6fdd7-3410-4823-b8e0-68f8a2ea1b7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:41.008748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6fdd7-3410-4823-b8e0-68f8a2ea1b7c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:41.026694 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 141/277] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:41.027688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8823057-b960-4216-9545-9e037ed18981 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:41.056437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:41.091868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 137/278] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:04:40 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:41.092722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6fdd7-3410-4823-b8e0-68f8a2ea1b7c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:41.116167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:41.155065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8823057-b960-4216-9545-9e037ed18981 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:41.197952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a0a5c-022b-4417-87b3-417b31931782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:41.199363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a0a5c-022b-4417-87b3-417b31931782 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.199954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a0a5c-022b-4417-87b3-417b31931782 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.200435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a0a5c-022b-4417-87b3-417b31931782 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.201695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a0a5c-022b-4417-87b3-417b31931782 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:41.202210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a0a5c-022b-4417-87b3-417b31931782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:41.203994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a0a5c-022b-4417-87b3-417b31931782 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:41.213816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6fdd7-3410-4823-b8e0-68f8a2ea1b7c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:41.216261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 142/279] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:41.222073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a0a5c-022b-4417-87b3-417b31931782 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:41.251758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82369439-e9b1-4ec8-9364-7f2262edee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:41.252442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82369439-e9b1-4ec8-9364-7f2262edee54 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.252740 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82369439-e9b1-4ec8-9364-7f2262edee54 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.253061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82369439-e9b1-4ec8-9364-7f2262edee54 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.264575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82369439-e9b1-4ec8-9364-7f2262edee54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-82369439-e9b1-4ec8-9364-7f2262edee54 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:41.270193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:41.271087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:41.272473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:41.273087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 138/280] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:41.278245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82369439-e9b1-4ec8-9364-7f2262edee54 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:41.321233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:41.341411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a0a5c-022b-4417-87b3-417b31931782 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:41.367307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:41.388595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf48e3-1288-420e-9891-79f3fa81ae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:41.389259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf48e3-1288-420e-9891-79f3fa81ae38 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.389607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf48e3-1288-420e-9891-79f3fa81ae38 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.389811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf48e3-1288-420e-9891-79f3fa81ae38 None admin] PATH_INFO: `/v3/users/5651d274279140c694dfe928d2a7ee78` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.394030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82369439-e9b1-4ec8-9364-7f2262edee54 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:41.408018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf48e3-1288-420e-9891-79f3fa81ae38 None admin] RBAC: Authorizing `identity:get_user(user_id=5651d274279140c694dfe928d2a7ee78)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:41.409502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf48e3-1288-420e-9891-79f3fa81ae38 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:41.411568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 143/281] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:41 2022] GET /identity/v3/users/5651d274279140c694dfe928d2a7ee78 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:41.413444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b961c18-6363-4316-bdbd-401b3c261f43 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.414067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b961c18-6363-4316-bdbd-401b3c261f43 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.414067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b961c18-6363-4316-bdbd-401b3c261f43 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.415229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 144/282] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:41.420768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b961c18-6363-4316-bdbd-401b3c261f43 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:41.440284 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03922426-6073-4b31-8769-545af1047e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:41.440931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03922426-6073-4b31-8769-545af1047e7d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.441191 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03922426-6073-4b31-8769-545af1047e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.441470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03922426-6073-4b31-8769-545af1047e7d None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.442194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03922426-6073-4b31-8769-545af1047e7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:41.443107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03922426-6073-4b31-8769-545af1047e7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:41.444429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03922426-6073-4b31-8769-545af1047e7d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:41.453556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 139/283] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:41.454820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97579e1-18a7-4cbc-9642-1367c9d43bdd None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.455114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97579e1-18a7-4cbc-9642-1367c9d43bdd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.455395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97579e1-18a7-4cbc-9642-1367c9d43bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.475721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a97579e1-18a7-4cbc-9642-1367c9d43bdd None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:41.476956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a97579e1-18a7-4cbc-9642-1367c9d43bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:41.547686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b961c18-6363-4316-bdbd-401b3c261f43 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:41.554485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 140/284] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:41.555563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97579e1-18a7-4cbc-9642-1367c9d43bdd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:41.609844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dd12b-1ed2-4d35-af26-b04561c648ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:41.610529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dd12b-1ed2-4d35-af26-b04561c648ed None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.610822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dd12b-1ed2-4d35-af26-b04561c648ed None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.611099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dd12b-1ed2-4d35-af26-b04561c648ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.615171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dd12b-1ed2-4d35-af26-b04561c648ed None admin] RBAC: Adding query filter params (role.id=512a743b8ca748f8b793681748be7906, user.id=5651d274279140c694dfe928d2a7ee78, scope.system=all) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:41.615525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dd12b-1ed2-4d35-af26-b04561c648ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:41.618389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dd12b-1ed2-4d35-af26-b04561c648ed None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:41.628012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 145/285] 10.222.0.25 () {56 vars in 1308 bytes} [Sat Aug 6 01:04:41 2022] GET /identity/v3/role_assignments?user.id=5651d274279140c694dfe928d2a7ee78&scope.system=all&role.id=512a743b8ca748f8b793681748be7906 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:41.629042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dd12b-1ed2-4d35-af26-b04561c648ed None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:41.631533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:41.634674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:41.666638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97579e1-18a7-4cbc-9642-1367c9d43bdd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:41.705321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907926c-4f55-4dec-9b0e-807635ad80c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:41.706372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907926c-4f55-4dec-9b0e-807635ad80c4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.706899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907926c-4f55-4dec-9b0e-807635ad80c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.707375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907926c-4f55-4dec-9b0e-807635ad80c4 None admin] PATH_INFO: `/v3/users/973fa099ee414f60b501373c88a8268f` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.708837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907926c-4f55-4dec-9b0e-807635ad80c4 None admin] RBAC: Authorizing `identity:get_user(user_id=973fa099ee414f60b501373c88a8268f)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:41.711043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907926c-4f55-4dec-9b0e-807635ad80c4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:41.713498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 141/286] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:41 2022] GET /identity/v3/users/973fa099ee414f60b501373c88a8268f => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:41.722936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907926c-4f55-4dec-9b0e-807635ad80c4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:41.778766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dd12b-1ed2-4d35-af26-b04561c648ed None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:41.836729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907926c-4f55-4dec-9b0e-807635ad80c4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:41.848427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde187ab-e01d-4242-b1af-14779a34b7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:41.849014 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde187ab-e01d-4242-b1af-14779a34b7dc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.849294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde187ab-e01d-4242-b1af-14779a34b7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.849592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde187ab-e01d-4242-b1af-14779a34b7dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.867831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde187ab-e01d-4242-b1af-14779a34b7dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-cde187ab-e01d-4242-b1af-14779a34b7dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:41.876404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:41.877586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:41.878414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 146/287] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:41.880360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5ca75-e7df-47c8-a8f6-160f6052ce1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:41.881531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5ca75-e7df-47c8-a8f6-160f6052ce1e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:41.881806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5ca75-e7df-47c8-a8f6-160f6052ce1e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:41.882080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5ca75-e7df-47c8-a8f6-160f6052ce1e None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:41.883729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5ca75-e7df-47c8-a8f6-160f6052ce1e None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:41.884459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde187ab-e01d-4242-b1af-14779a34b7dc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:41.885480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5ca75-e7df-47c8-a8f6-160f6052ce1e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:41.887596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 142/288] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:41 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:41.894135 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5ca75-e7df-47c8-a8f6-160f6052ce1e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:41.906690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:42.004144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:42.073926 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde187ab-e01d-4242-b1af-14779a34b7dc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:42.084874 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5ca75-e7df-47c8-a8f6-160f6052ce1e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:42.135138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c3f64-5fb6-4c06-895d-a5e2e088c2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:42.135732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c3f64-5fb6-4c06-895d-a5e2e088c2bf None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:42.136035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c3f64-5fb6-4c06-895d-a5e2e088c2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:42.136371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c3f64-5fb6-4c06-895d-a5e2e088c2bf None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53/users/973fa099ee414f60b501373c88a8268f/roles/324a803dd87d4027b73d871fe46ea228` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:42.137888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d323fa-049f-4d9f-9300-db6f0f85e4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:42.138247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c3f64-5fb6-4c06-895d-a5e2e088c2bf None admin] RBAC: Authorizing `identity:create_grant(project_id=3059810e56d143ab91728e2220010d53, user_id=973fa099ee414f60b501373c88a8268f, role_id=324a803dd87d4027b73d871fe46ea228)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:42.138520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d323fa-049f-4d9f-9300-db6f0f85e4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:42.138806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d323fa-049f-4d9f-9300-db6f0f85e4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:42.139083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d323fa-049f-4d9f-9300-db6f0f85e4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:42.139686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c3f64-5fb6-4c06-895d-a5e2e088c2bf None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:42.139815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d323fa-049f-4d9f-9300-db6f0f85e4c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:42.140132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d323fa-049f-4d9f-9300-db6f0f85e4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:42.141517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d323fa-049f-4d9f-9300-db6f0f85e4c5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:42.155978 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 147/289] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:42.161228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d323fa-049f-4d9f-9300-db6f0f85e4c5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:42.216288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 143/290] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:04:41 2022] PUT /identity/v3/projects/3059810e56d143ab91728e2220010d53/users/973fa099ee414f60b501373c88a8268f/roles/324a803dd87d4027b73d871fe46ea228 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:04:42.291111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d323fa-049f-4d9f-9300-db6f0f85e4c5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:42.321884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:42.333100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7722c19-fc13-4266-94e3-cc0c99ff8b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:42.334976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7722c19-fc13-4266-94e3-cc0c99ff8b16 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:42.334976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7722c19-fc13-4266-94e3-cc0c99ff8b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:42.334976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7722c19-fc13-4266-94e3-cc0c99ff8b16 None admin] PATH_INFO: `/v3/users/9af37e8dc107403988328caa32a19828` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:42.336084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7722c19-fc13-4266-94e3-cc0c99ff8b16 None admin] RBAC: Authorizing `identity:get_user(user_id=9af37e8dc107403988328caa32a19828)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:42.337658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7722c19-fc13-4266-94e3-cc0c99ff8b16 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:42.339603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 148/291] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:42 2022] GET /identity/v3/users/9af37e8dc107403988328caa32a19828 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:42.345288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c3f64-5fb6-4c06-895d-a5e2e088c2bf None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:42.453823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:42.491426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c3f64-5fb6-4c06-895d-a5e2e088c2bf None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:42.529527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa5d2d-721a-47e9-a3da-c5a540c749ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:42.530196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa5d2d-721a-47e9-a3da-c5a540c749ba None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:42.530485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa5d2d-721a-47e9-a3da-c5a540c749ba None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:42.530758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa5d2d-721a-47e9-a3da-c5a540c749ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:42.621041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa5d2d-721a-47e9-a3da-c5a540c749ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:42.622549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa5d2d-721a-47e9-a3da-c5a540c749ba None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:42.725298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:42.769550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 144/292] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:42.775650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7722c19-fc13-4266-94e3-cc0c99ff8b16 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:42.934397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7722c19-fc13-4266-94e3-cc0c99ff8b16 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:42.977047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7178bc-3531-4aeb-9b23-f65e7a9b9306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:42.977621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7178bc-3531-4aeb-9b23-f65e7a9b9306 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:42.977904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7178bc-3531-4aeb-9b23-f65e7a9b9306 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:42.978183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7178bc-3531-4aeb-9b23-f65e7a9b9306 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:42.984533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7178bc-3531-4aeb-9b23-f65e7a9b9306 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:42.986007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7178bc-3531-4aeb-9b23-f65e7a9b9306 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:42.988029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 149/293] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:42 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:42.993874 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa5d2d-721a-47e9-a3da-c5a540c749ba None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:43.119869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa5d2d-721a-47e9-a3da-c5a540c749ba None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:43.157650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe253d5a-edc2-4f94-8609-d1801b6ee521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:43.158277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253d5a-edc2-4f94-8609-d1801b6ee521 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:43.158559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253d5a-edc2-4f94-8609-d1801b6ee521 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:43.158832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253d5a-edc2-4f94-8609-d1801b6ee521 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:43.159677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe253d5a-edc2-4f94-8609-d1801b6ee521 None admin] RBAC: Adding query filter params (role.id=512a743b8ca748f8b793681748be7906, scope.project.id=3059810e56d143ab91728e2220010d53, user.id=9af37e8dc107403988328caa32a19828) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:43.160022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe253d5a-edc2-4f94-8609-d1801b6ee521 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:43.161469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe253d5a-edc2-4f94-8609-d1801b6ee521 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:43.168039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 145/294] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:42 2022] GET /identity/v3/role_assignments?user.id=9af37e8dc107403988328caa32a19828&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=512a743b8ca748f8b793681748be7906 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:43.642429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77859a33-4fe8-4a6c-b378-515c152f9a4f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:43.642429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77859a33-4fe8-4a6c-b378-515c152f9a4f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:43.642429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77859a33-4fe8-4a6c-b378-515c152f9a4f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:43.643435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 150/295] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:43.648880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b1ed-ea21-4d47-aa8e-82f1e1e455b1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:43.649283 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b1ed-ea21-4d47-aa8e-82f1e1e455b1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:43.649456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b1ed-ea21-4d47-aa8e-82f1e1e455b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:43.672332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-16d7b1ed-ea21-4d47-aa8e-82f1e1e455b1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:43.673669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-16d7b1ed-ea21-4d47-aa8e-82f1e1e455b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:43.708807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:43.759926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 146/296] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:43.782614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77859a33-4fe8-4a6c-b378-515c152f9a4f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:43.830361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd845680-0663-47c3-9c2c-e62f4b685d96 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:43.830902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd845680-0663-47c3-9c2c-e62f4b685d96 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:43.831295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd845680-0663-47c3-9c2c-e62f4b685d96 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:43.836703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 147/297] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:43.838102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456853-f622-4531-81c0-1688b45cd80e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:43.838388 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456853-f622-4531-81c0-1688b45cd80e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:43.838659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456853-f622-4531-81c0-1688b45cd80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:43.854075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:43.856723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-72456853-f622-4531-81c0-1688b45cd80e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:43.857901 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-72456853-f622-4531-81c0-1688b45cd80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:43.903636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77859a33-4fe8-4a6c-b378-515c152f9a4f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:43.929488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 148/298] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:43.949498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920fda0-4ea0-45cf-a907-801250b3cdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:43.950487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920fda0-4ea0-45cf-a907-801250b3cdbb None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:43.951235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920fda0-4ea0-45cf-a907-801250b3cdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:43.951710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920fda0-4ea0-45cf-a907-801250b3cdbb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:43.953204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920fda0-4ea0-45cf-a907-801250b3cdbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:43.954379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72456853-f622-4531-81c0-1688b45cd80e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:43.958711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920fda0-4ea0-45cf-a907-801250b3cdbb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:43.961036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 151/299] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:43.968696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920fda0-4ea0-45cf-a907-801250b3cdbb None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:44.050292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:44.061493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72456853-f622-4531-81c0-1688b45cd80e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:44.098443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198cd4e-8454-4e22-b808-292f282daf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:44.099084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198cd4e-8454-4e22-b808-292f282daf9a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.099367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198cd4e-8454-4e22-b808-292f282daf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.100197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198cd4e-8454-4e22-b808-292f282daf9a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.102516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920fda0-4ea0-45cf-a907-801250b3cdbb None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:44.110241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198cd4e-8454-4e22-b808-292f282daf9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9198cd4e-8454-4e22-b808-292f282daf9a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:44.115711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:44.116620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:44.117436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:44.121614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 149/300] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:44.122781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c8a47-4549-4d43-adf1-f29767ea4446 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.123074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c8a47-4549-4d43-adf1-f29767ea4446 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.123348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c8a47-4549-4d43-adf1-f29767ea4446 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.129311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 150/301] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:44.129311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c8a47-4549-4d43-adf1-f29767ea4446 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:44.139395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:44.145687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f417d4d-18c6-4a2c-b81a-ef3ad291d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:44.146205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f417d4d-18c6-4a2c-b81a-ef3ad291d48e None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.146527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f417d4d-18c6-4a2c-b81a-ef3ad291d48e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.146769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f417d4d-18c6-4a2c-b81a-ef3ad291d48e None admin] PATH_INFO: `/v3/groups` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.147424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f417d4d-18c6-4a2c-b81a-ef3ad291d48e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:44.148899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f417d4d-18c6-4a2c-b81a-ef3ad291d48e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:44.238010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 152/302] 10.222.0.25 () {60 vars in 1134 bytes} [Sat Aug 6 01:04:43 2022] POST /identity/v3/groups => generated 229 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:44.238010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1b971-e51a-4a84-816b-9ab2b53fcd7f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.238010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1b971-e51a-4a84-816b-9ab2b53fcd7f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.238010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1b971-e51a-4a84-816b-9ab2b53fcd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.240251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c8a47-4549-4d43-adf1-f29767ea4446 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:44.256384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6bb1b971-e51a-4a84-816b-9ab2b53fcd7f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:44.257880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6bb1b971-e51a-4a84-816b-9ab2b53fcd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:44.291285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822314c-be4f-4113-a069-526c28ea352e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:44.291971 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822314c-be4f-4113-a069-526c28ea352e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.292253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822314c-be4f-4113-a069-526c28ea352e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.292529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822314c-be4f-4113-a069-526c28ea352e None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.293540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822314c-be4f-4113-a069-526c28ea352e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:44.294291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822314c-be4f-4113-a069-526c28ea352e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:44.295983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822314c-be4f-4113-a069-526c28ea352e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:44.306672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 151/303] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:44.312322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822314c-be4f-4113-a069-526c28ea352e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:44.328903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:44.347809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 153/304] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:44.366571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1b971-e51a-4a84-816b-9ab2b53fcd7f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:44.444113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822314c-be4f-4113-a069-526c28ea352e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:44.491290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae279e-e7df-444a-9d65-63b2cd78f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:44.492317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae279e-e7df-444a-9d65-63b2cd78f519 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.492777 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1b971-e51a-4a84-816b-9ab2b53fcd7f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:44.492946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae279e-e7df-444a-9d65-63b2cd78f519 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.493386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae279e-e7df-444a-9d65-63b2cd78f519 None admin] PATH_INFO: `/v3/users/e67adddace0844b6b0530f3c4a869a64` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.511199 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae279e-e7df-444a-9d65-63b2cd78f519 None admin] RBAC: Authorizing `identity:get_user(user_id=e67adddace0844b6b0530f3c4a869a64)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:44.513417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae279e-e7df-444a-9d65-63b2cd78f519 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:44.515767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 152/305] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:44 2022] GET /identity/v3/users/e67adddace0844b6b0530f3c4a869a64 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:44.519823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:44.522360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae279e-e7df-444a-9d65-63b2cd78f519 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:44.531124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a06fdb-17cb-430d-b8cc-2b7f76ce82b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:44.532079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a06fdb-17cb-430d-b8cc-2b7f76ce82b3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.532498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a06fdb-17cb-430d-b8cc-2b7f76ce82b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.532996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a06fdb-17cb-430d-b8cc-2b7f76ce82b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.547464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a06fdb-17cb-430d-b8cc-2b7f76ce82b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d0a06fdb-17cb-430d-b8cc-2b7f76ce82b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:44.555227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:44.556310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:44.557366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:44.559568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 154/306] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:44.567075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a06fdb-17cb-430d-b8cc-2b7f76ce82b3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:44.614752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:44.648355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae279e-e7df-444a-9d65-63b2cd78f519 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:44.680651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a06fdb-17cb-430d-b8cc-2b7f76ce82b3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:44.682371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6e387-abb5-4f7d-b7e5-2da8138385d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:44.682995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6e387-abb5-4f7d-b7e5-2da8138385d0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.683270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6e387-abb5-4f7d-b7e5-2da8138385d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.683541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6e387-abb5-4f7d-b7e5-2da8138385d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.684368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6e387-abb5-4f7d-b7e5-2da8138385d0 None admin] RBAC: Adding query filter params (role.id=324a803dd87d4027b73d871fe46ea228, user.id=e67adddace0844b6b0530f3c4a869a64, scope.system=all) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:44.684676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6e387-abb5-4f7d-b7e5-2da8138385d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:44.686095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6e387-abb5-4f7d-b7e5-2da8138385d0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:44.691189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 153/307] 10.222.0.25 () {56 vars in 1308 bytes} [Sat Aug 6 01:04:44 2022] GET /identity/v3/role_assignments?user.id=e67adddace0844b6b0530f3c4a869a64&scope.system=all&role.id=324a803dd87d4027b73d871fe46ea228 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:44.692519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d158-c88e-468c-97a1-4d1c9a29193f None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.692804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d158-c88e-468c-97a1-4d1c9a29193f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.693077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d158-c88e-468c-97a1-4d1c9a29193f None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.698985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 154/308] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:44.700486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bcedc-005c-4fd2-bc9b-6805d28a3d5c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.701029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bcedc-005c-4fd2-bc9b-6805d28a3d5c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.701301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bcedc-005c-4fd2-bc9b-6805d28a3d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.727034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7a8bcedc-005c-4fd2-bc9b-6805d28a3d5c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:44.729468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7a8bcedc-005c-4fd2-bc9b-6805d28a3d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:44.732536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbed64-65e1-4a0c-9812-54b7a8ea4025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:44.733296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbed64-65e1-4a0c-9812-54b7a8ea4025 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.733674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbed64-65e1-4a0c-9812-54b7a8ea4025 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.734731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbed64-65e1-4a0c-9812-54b7a8ea4025 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.734954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbed64-65e1-4a0c-9812-54b7a8ea4025 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:44.735358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbed64-65e1-4a0c-9812-54b7a8ea4025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:44.737035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbed64-65e1-4a0c-9812-54b7a8ea4025 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:44.747586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 155/309] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:44.752907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbed64-65e1-4a0c-9812-54b7a8ea4025 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:44.812271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:44.829522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 155/310] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:44.852602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bcedc-005c-4fd2-bc9b-6805d28a3d5c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:44.884770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbed64-65e1-4a0c-9812-54b7a8ea4025 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:44.921053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd87cf0-2348-4678-92b9-5c1ab7b38c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:44.921652 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd87cf0-2348-4678-92b9-5c1ab7b38c93 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:44.921936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd87cf0-2348-4678-92b9-5c1ab7b38c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:44.922213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd87cf0-2348-4678-92b9-5c1ab7b38c93 None admin] PATH_INFO: `/v3/users/5651d274279140c694dfe928d2a7ee78` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:44.923392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd87cf0-2348-4678-92b9-5c1ab7b38c93 None admin] RBAC: Authorizing `identity:get_user(user_id=5651d274279140c694dfe928d2a7ee78)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:44.925179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd87cf0-2348-4678-92b9-5c1ab7b38c93 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:44.931595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 156/311] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:44 2022] GET /identity/v3/users/5651d274279140c694dfe928d2a7ee78 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:44.932925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd87cf0-2348-4678-92b9-5c1ab7b38c93 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:44.934769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:44.981713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bcedc-005c-4fd2-bc9b-6805d28a3d5c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:45.026849 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dadaf-9081-4fc1-89ca-c1a994468b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:45.027768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dadaf-9081-4fc1-89ca-c1a994468b4a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:45.028072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dadaf-9081-4fc1-89ca-c1a994468b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:45.028344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dadaf-9081-4fc1-89ca-c1a994468b4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:45.046121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68dadaf-9081-4fc1-89ca-c1a994468b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:45.049083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd87cf0-2348-4678-92b9-5c1ab7b38c93 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-c68dadaf-9081-4fc1-89ca-c1a994468b4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:45.051904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:45.053341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:45.054136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:45.061433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:45.061995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 156/312] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:45.063697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dadaf-9081-4fc1-89ca-c1a994468b4a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:45.090702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb832ca8-4684-40b5-b266-7a8f50f96d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:45.091682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb832ca8-4684-40b5-b266-7a8f50f96d5c None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:45.092002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb832ca8-4684-40b5-b266-7a8f50f96d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:45.092283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb832ca8-4684-40b5-b266-7a8f50f96d5c None admin] PATH_INFO: `/v3/system/users/5651d274279140c694dfe928d2a7ee78/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:45.094163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb832ca8-4684-40b5-b266-7a8f50f96d5c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5651d274279140c694dfe928d2a7ee78, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:45.096172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb832ca8-4684-40b5-b266-7a8f50f96d5c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:45.169114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dadaf-9081-4fc1-89ca-c1a994468b4a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:45.179261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 157/313] 10.222.0.25 () {58 vars in 1334 bytes} [Sat Aug 6 01:04:44 2022] PUT /identity/v3/system/users/5651d274279140c694dfe928d2a7ee78/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:45.202132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:45.214550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c857e-fae9-4378-9133-73bcf28a64a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:45.215381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c857e-fae9-4378-9133-73bcf28a64a5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:45.215680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c857e-fae9-4378-9133-73bcf28a64a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:45.216010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c857e-fae9-4378-9133-73bcf28a64a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:45.216753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c857e-fae9-4378-9133-73bcf28a64a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:45.217061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c857e-fae9-4378-9133-73bcf28a64a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:45.218452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c857e-fae9-4378-9133-73bcf28a64a5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:45.228175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 157/314] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:45.234377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb832ca8-4684-40b5-b266-7a8f50f96d5c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:45.351115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb832ca8-4684-40b5-b266-7a8f50f96d5c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:45.388204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6a587-cd6e-4a97-9b91-78b3059d85d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:45.388865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6a587-cd6e-4a97-9b91-78b3059d85d3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:45.389152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6a587-cd6e-4a97-9b91-78b3059d85d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:45.389430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6a587-cd6e-4a97-9b91-78b3059d85d3 None admin] PATH_INFO: `/v3/users/973fa099ee414f60b501373c88a8268f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:45.390608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6a587-cd6e-4a97-9b91-78b3059d85d3 None admin] RBAC: Authorizing `identity:get_user(user_id=973fa099ee414f60b501373c88a8268f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:45.392639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6a587-cd6e-4a97-9b91-78b3059d85d3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:45.394043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 158/315] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:45 2022] GET /identity/v3/users/973fa099ee414f60b501373c88a8268f => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:45.399262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c857e-fae9-4378-9133-73bcf28a64a5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:45.510558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c857e-fae9-4378-9133-73bcf28a64a5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:45.544513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30490b9-651c-4bd7-8a7f-fb56e6f69686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:45.545133 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30490b9-651c-4bd7-8a7f-fb56e6f69686 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:45.545411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30490b9-651c-4bd7-8a7f-fb56e6f69686 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:45.545690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30490b9-651c-4bd7-8a7f-fb56e6f69686 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:45.601285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:45.614435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30490b9-651c-4bd7-8a7f-fb56e6f69686 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:45.616636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30490b9-651c-4bd7-8a7f-fb56e6f69686 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:45.725935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 158/316] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:45.731937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6a587-cd6e-4a97-9b91-78b3059d85d3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:45.777115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:45.835952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6a587-cd6e-4a97-9b91-78b3059d85d3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:45.875226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c718c-7ce5-4e9d-98c3-2e7e392d2ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:45.875975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c718c-7ce5-4e9d-98c3-2e7e392d2ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:45.876265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c718c-7ce5-4e9d-98c3-2e7e392d2ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:45.876544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c718c-7ce5-4e9d-98c3-2e7e392d2ed3 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:45.877712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c718c-7ce5-4e9d-98c3-2e7e392d2ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:45.879180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c718c-7ce5-4e9d-98c3-2e7e392d2ed3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:45.881069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 159/317] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:45 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:45.886770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30490b9-651c-4bd7-8a7f-fb56e6f69686 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:45.977864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:45.993168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30490b9-651c-4bd7-8a7f-fb56e6f69686 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:46.028512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405854e0-e2d8-43f5-b63e-3bef97a67297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:46.029055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405854e0-e2d8-43f5-b63e-3bef97a67297 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:46.029331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405854e0-e2d8-43f5-b63e-3bef97a67297 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:46.029602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405854e0-e2d8-43f5-b63e-3bef97a67297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:46.030384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405854e0-e2d8-43f5-b63e-3bef97a67297 None admin] RBAC: Adding query filter params (role.id=324a803dd87d4027b73d871fe46ea228, scope.project.id=3059810e56d143ab91728e2220010d53, user.id=973fa099ee414f60b501373c88a8268f) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:46.030690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405854e0-e2d8-43f5-b63e-3bef97a67297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:46.032138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405854e0-e2d8-43f5-b63e-3bef97a67297 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:46.039176 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 159/318] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:04:45 2022] GET /identity/v3/role_assignments?user.id=973fa099ee414f60b501373c88a8268f&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=324a803dd87d4027b73d871fe46ea228 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:46.354465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21704d5f-a863-4d36-bdbb-04c71c014b3e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:46.354803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21704d5f-a863-4d36-bdbb-04c71c014b3e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:46.354919 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21704d5f-a863-4d36-bdbb-04c71c014b3e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:46.356421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 160/319] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:46.361498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7c03e-5538-4518-a2a5-63aed00c8143 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:46.361787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7c03e-5538-4518-a2a5-63aed00c8143 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:46.362064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7c03e-5538-4518-a2a5-63aed00c8143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:46.380366 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b9a7c03e-5538-4518-a2a5-63aed00c8143 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:46.382132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b9a7c03e-5538-4518-a2a5-63aed00c8143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:46.447875 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 160/320] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:46.470663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21704d5f-a863-4d36-bdbb-04c71c014b3e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:46.570391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21704d5f-a863-4d36-bdbb-04c71c014b3e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:46.583222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:46.608890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315f70a-638c-4de3-ba16-a790b0a1c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:46.609467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315f70a-638c-4de3-ba16-a790b0a1c603 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:46.609749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315f70a-638c-4de3-ba16-a790b0a1c603 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:46.610025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315f70a-638c-4de3-ba16-a790b0a1c603 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:46.611151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315f70a-638c-4de3-ba16-a790b0a1c603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:46.612698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315f70a-638c-4de3-ba16-a790b0a1c603 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:46.614525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 161/321] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:46.620977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7c03e-5538-4518-a2a5-63aed00c8143 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:46.705721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:46.724414 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7c03e-5538-4518-a2a5-63aed00c8143 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:46.758273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b85cca-e08b-4d0c-a404-cf4ff2e72efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:46.758988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85cca-e08b-4d0c-a404-cf4ff2e72efc None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:46.759185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85cca-e08b-4d0c-a404-cf4ff2e72efc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:46.759444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b85cca-e08b-4d0c-a404-cf4ff2e72efc None admin] PATH_INFO: `/v3/groups` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:46.760146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b85cca-e08b-4d0c-a404-cf4ff2e72efc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:46.761575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b85cca-e08b-4d0c-a404-cf4ff2e72efc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:46.861717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e771f50-8a0d-44df-a2ee-d3edde0220c9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:46.862206 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e771f50-8a0d-44df-a2ee-d3edde0220c9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:46.862293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e771f50-8a0d-44df-a2ee-d3edde0220c9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:46.863669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 162/322] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:46.864054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 161/323] 10.222.0.25 () {60 vars in 1134 bytes} [Sat Aug 6 01:04:46 2022] POST /identity/v3/groups => generated 226 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:04:46.869436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea1e74-79df-4e5e-93df-b11b612019e2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:46.869726 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea1e74-79df-4e5e-93df-b11b612019e2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:46.870004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea1e74-79df-4e5e-93df-b11b612019e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:46.889875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-50ea1e74-79df-4e5e-93df-b11b612019e2 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:46.891146 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-50ea1e74-79df-4e5e-93df-b11b612019e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:46.965449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 163/324] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:46.987560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b85cca-e08b-4d0c-a404-cf4ff2e72efc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:47.093648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b85cca-e08b-4d0c-a404-cf4ff2e72efc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:47.129916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e0acb-c5dd-420b-a72e-8f2c01fc7029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:47.130506 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0acb-c5dd-420b-a72e-8f2c01fc7029 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:47.131398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0acb-c5dd-420b-a72e-8f2c01fc7029 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:47.131674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0acb-c5dd-420b-a72e-8f2c01fc7029 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:47.140594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:47.145454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e0acb-c5dd-420b-a72e-8f2c01fc7029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-750e0acb-c5dd-420b-a72e-8f2c01fc7029 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:47.151081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:47.152037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:47.152884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 162/325] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:47.159574 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea1e74-79df-4e5e-93df-b11b612019e2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:47.276607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea1e74-79df-4e5e-93df-b11b612019e2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:47.282388 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375afc6-ec4a-4943-8521-699b06749014 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:47.282955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375afc6-ec4a-4943-8521-699b06749014 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:47.283238 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375afc6-ec4a-4943-8521-699b06749014 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:47.284820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 163/326] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:47.287923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:47.293042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033462e-a3d0-42ff-b3bc-85f11b23ff9b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:47.293042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033462e-a3d0-42ff-b3bc-85f11b23ff9b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:47.293042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033462e-a3d0-42ff-b3bc-85f11b23ff9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:47.313277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9033462e-a3d0-42ff-b3bc-85f11b23ff9b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:47.314528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9033462e-a3d0-42ff-b3bc-85f11b23ff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:47.325015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fb80e-5674-40ec-9840-d79d84627539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:47.325654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fb80e-5674-40ec-9840-d79d84627539 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:47.326205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fb80e-5674-40ec-9840-d79d84627539 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:47.326485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fb80e-5674-40ec-9840-d79d84627539 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:47.327211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fb80e-5674-40ec-9840-d79d84627539 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:47.327520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fb80e-5674-40ec-9840-d79d84627539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:47.328950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fb80e-5674-40ec-9840-d79d84627539 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:47.342284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 164/327] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:47.343461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fb80e-5674-40ec-9840-d79d84627539 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:47.430218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 164/328] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:47.454595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fb80e-5674-40ec-9840-d79d84627539 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:47.469237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033462e-a3d0-42ff-b3bc-85f11b23ff9b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:47.489954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb64c31-7d22-40d2-8f14-1920365f1afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:47.491200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb64c31-7d22-40d2-8f14-1920365f1afe None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:47.491649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb64c31-7d22-40d2-8f14-1920365f1afe None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:47.491963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb64c31-7d22-40d2-8f14-1920365f1afe None admin] PATH_INFO: `/v3/users/e67adddace0844b6b0530f3c4a869a64` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:47.493655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb64c31-7d22-40d2-8f14-1920365f1afe None admin] RBAC: Authorizing `identity:get_user(user_id=e67adddace0844b6b0530f3c4a869a64)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:47.495622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb64c31-7d22-40d2-8f14-1920365f1afe None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:47.497936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 165/329] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:47 2022] GET /identity/v3/users/e67adddace0844b6b0530f3c4a869a64 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:47.504792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb64c31-7d22-40d2-8f14-1920365f1afe None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:47.557368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:47.611180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033462e-a3d0-42ff-b3bc-85f11b23ff9b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:47.671764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:47.672750 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb64c31-7d22-40d2-8f14-1920365f1afe None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:47.685722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3fa3b-0dfb-42ad-91cd-5826957c9627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:47.686341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3fa3b-0dfb-42ad-91cd-5826957c9627 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:47.686622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3fa3b-0dfb-42ad-91cd-5826957c9627 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:47.686892 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3fa3b-0dfb-42ad-91cd-5826957c9627 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:47.709934 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3fa3b-0dfb-42ad-91cd-5826957c9627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-cdd3fa3b-0dfb-42ad-91cd-5826957c9627 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:47.715662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:47.716660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:47.717464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:47.723806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 165/330] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:47.730202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3fa3b-0dfb-42ad-91cd-5826957c9627 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:47.751683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b8523-4911-4297-9692-c23831f857f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:47.753150 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b8523-4911-4297-9692-c23831f857f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:47.753664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b8523-4911-4297-9692-c23831f857f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:47.754356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b8523-4911-4297-9692-c23831f857f0 None admin] PATH_INFO: `/v3/system/users/e67adddace0844b6b0530f3c4a869a64/roles/324a803dd87d4027b73d871fe46ea228` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:47.756688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b8523-4911-4297-9692-c23831f857f0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e67adddace0844b6b0530f3c4a869a64, role_id=324a803dd87d4027b73d871fe46ea228)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:47.758841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b8523-4911-4297-9692-c23831f857f0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:47.830842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 166/331] 10.222.0.25 () {58 vars in 1334 bytes} [Sat Aug 6 01:04:47 2022] PUT /identity/v3/system/users/e67adddace0844b6b0530f3c4a869a64/roles/324a803dd87d4027b73d871fe46ea228 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:04:47.859705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3fa3b-0dfb-42ad-91cd-5826957c9627 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:47.903408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f6a97-c874-4c62-95f2-f350ec577bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:47.904126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f6a97-c874-4c62-95f2-f350ec577bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:47.904415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f6a97-c874-4c62-95f2-f350ec577bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:47.904747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f6a97-c874-4c62-95f2-f350ec577bb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:47.905492 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f6a97-c874-4c62-95f2-f350ec577bb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:47.905801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f6a97-c874-4c62-95f2-f350ec577bb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:47.907270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f6a97-c874-4c62-95f2-f350ec577bb3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:47.917444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 166/332] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:47.923240 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b8523-4911-4297-9692-c23831f857f0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:48.114225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b8523-4911-4297-9692-c23831f857f0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:48.115934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:48.151699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7f1fc-9c90-4c9d-9f4a-af5ea3b765bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:48.152420 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7f1fc-9c90-4c9d-9f4a-af5ea3b765bc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:48.152703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7f1fc-9c90-4c9d-9f4a-af5ea3b765bc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:48.153046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7f1fc-9c90-4c9d-9f4a-af5ea3b765bc None admin] PATH_INFO: `/v3/users/5651d274279140c694dfe928d2a7ee78` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:48.156263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7f1fc-9c90-4c9d-9f4a-af5ea3b765bc None admin] RBAC: Authorizing `identity:get_user(user_id=5651d274279140c694dfe928d2a7ee78)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:48.157825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7f1fc-9c90-4c9d-9f4a-af5ea3b765bc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:48.159790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 167/333] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:47 2022] GET /identity/v3/users/5651d274279140c694dfe928d2a7ee78 => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:48.166100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f6a97-c874-4c62-95f2-f350ec577bb3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:48.225075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:48.273006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f6a97-c874-4c62-95f2-f350ec577bb3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:48.307764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863bd8f-02b4-4893-b3bc-99d589d09316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:48.308482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863bd8f-02b4-4893-b3bc-99d589d09316 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:48.309069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863bd8f-02b4-4893-b3bc-99d589d09316 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:48.309396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863bd8f-02b4-4893-b3bc-99d589d09316 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:48.310195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863bd8f-02b4-4893-b3bc-99d589d09316 None admin] RBAC: Adding query filter params (role.id=512a743b8ca748f8b793681748be7906, user.id=5651d274279140c694dfe928d2a7ee78, scope.system=all) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:48.310507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863bd8f-02b4-4893-b3bc-99d589d09316 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:48.311988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863bd8f-02b4-4893-b3bc-99d589d09316 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:48.325100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 167/334] 10.222.0.25 () {56 vars in 1308 bytes} [Sat Aug 6 01:04:48 2022] GET /identity/v3/role_assignments?user.id=5651d274279140c694dfe928d2a7ee78&scope.system=all&role.id=512a743b8ca748f8b793681748be7906 => generated 510 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:49.676204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905b662-c4b4-4649-b474-3d6a725d8e0f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:49.676697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905b662-c4b4-4649-b474-3d6a725d8e0f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:49.676772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905b662-c4b4-4649-b474-3d6a725d8e0f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:49.680392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 168/335] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:49.687796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a482f2-41df-428b-84ff-4bc6d7acffd2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:49.688727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a482f2-41df-428b-84ff-4bc6d7acffd2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:49.689211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a482f2-41df-428b-84ff-4bc6d7acffd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:49.716833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c2a482f2-41df-428b-84ff-4bc6d7acffd2 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:49.718441 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c2a482f2-41df-428b-84ff-4bc6d7acffd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:49.808986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 168/336] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:49.841078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905b662-c4b4-4649-b474-3d6a725d8e0f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:49.914711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dec74f-9eb3-4cad-943a-fe6d76ad1a49 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:49.915033 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dec74f-9eb3-4cad-943a-fe6d76ad1a49 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:49.915272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dec74f-9eb3-4cad-943a-fe6d76ad1a49 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:49.917018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 169/337] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:49.922507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846e766-5c11-4c76-b48a-2494ef620e61 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:49.922800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846e766-5c11-4c76-b48a-2494ef620e61 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:49.923084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846e766-5c11-4c76-b48a-2494ef620e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:49.948787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8846e766-5c11-4c76-b48a-2494ef620e61 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:49.950085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8846e766-5c11-4c76-b48a-2494ef620e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:49.998935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:50.057447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905b662-c4b4-4649-b474-3d6a725d8e0f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:50.059060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 170/338] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:50.100691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8e7db-26ae-4ebd-99c4-8307c2b287ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:50.101702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8e7db-26ae-4ebd-99c4-8307c2b287ce None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:50.102251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8e7db-26ae-4ebd-99c4-8307c2b287ce None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:50.102724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8e7db-26ae-4ebd-99c4-8307c2b287ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:50.110364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846e766-5c11-4c76-b48a-2494ef620e61 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:50.119869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8e7db-26ae-4ebd-99c4-8307c2b287ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e3b8e7db-26ae-4ebd-99c4-8307c2b287ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:50.125262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:50.126156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:50.126972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:50.129178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 169/339] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:50.135565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8e7db-26ae-4ebd-99c4-8307c2b287ce None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:50.185902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:50.273197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8e7db-26ae-4ebd-99c4-8307c2b287ce None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:50.292002 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846e766-5c11-4c76-b48a-2494ef620e61 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:50.331760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6a060-cb8d-4037-900d-b48cd8e3b226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:50.332447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6a060-cb8d-4037-900d-b48cd8e3b226 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:50.332731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6a060-cb8d-4037-900d-b48cd8e3b226 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:50.333009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6a060-cb8d-4037-900d-b48cd8e3b226 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:50.340486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b6182-d7b1-4a9c-b583-09ef8cbc7d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:50.341225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b6182-d7b1-4a9c-b583-09ef8cbc7d92 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:50.341623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b6182-d7b1-4a9c-b583-09ef8cbc7d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:50.341909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b6182-d7b1-4a9c-b583-09ef8cbc7d92 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:50.342634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b6182-d7b1-4a9c-b583-09ef8cbc7d92 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:50.342945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b6182-d7b1-4a9c-b583-09ef8cbc7d92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:50.344270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6a060-cb8d-4037-900d-b48cd8e3b226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:50.344387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b6182-d7b1-4a9c-b583-09ef8cbc7d92 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-76d6a060-cb8d-4037-900d-b48cd8e3b226 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:50.349235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:50.350748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:50.351560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 171/340] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:50.352714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0622ba-7a43-4b9d-a2f0-ad07b78da303 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:50.352868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0622ba-7a43-4b9d-a2f0-ad07b78da303 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:50.353034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0622ba-7a43-4b9d-a2f0-ad07b78da303 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:50.354900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 172/341] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:50.360131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c854c6-4d69-4e97-b763-572487f3bbe3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:50.360419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c854c6-4d69-4e97-b763-572487f3bbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:50.360695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c854c6-4d69-4e97-b763-572487f3bbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:50.364323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 170/342] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:50.365300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b6182-d7b1-4a9c-b583-09ef8cbc7d92 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:50.387577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-51c854c6-4d69-4e97-b763-572487f3bbe3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:50.389883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-51c854c6-4d69-4e97-b763-572487f3bbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:50.424134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:50.469887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 173/343] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:50.470757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c854c6-4d69-4e97-b763-572487f3bbe3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:50.595053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c854c6-4d69-4e97-b763-572487f3bbe3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:50.632239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:50.634941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f8b89-3c44-4fc4-b253-8d13837bfc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:50.635579 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f8b89-3c44-4fc4-b253-8d13837bfc82 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:50.635902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f8b89-3c44-4fc4-b253-8d13837bfc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:50.636184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f8b89-3c44-4fc4-b253-8d13837bfc82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:50.639539 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b6182-d7b1-4a9c-b583-09ef8cbc7d92 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:50.666274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:50.692584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07681156-8f63-4fcc-8e4b-69fc23749f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:50.693776 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681156-8f63-4fcc-8e4b-69fc23749f37 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:50.695448 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681156-8f63-4fcc-8e4b-69fc23749f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:50.695448 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681156-8f63-4fcc-8e4b-69fc23749f37 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:50.695448 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07681156-8f63-4fcc-8e4b-69fc23749f37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:50.695448 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07681156-8f63-4fcc-8e4b-69fc23749f37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:50.696861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07681156-8f63-4fcc-8e4b-69fc23749f37 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:50.710671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 171/344] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:50.711049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f8b89-3c44-4fc4-b253-8d13837bfc82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:50.711558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07681156-8f63-4fcc-8e4b-69fc23749f37 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:50.712664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f8b89-3c44-4fc4-b253-8d13837bfc82 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:50.822246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 174/345] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:50.823266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f8b89-3c44-4fc4-b253-8d13837bfc82 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:50.873851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07681156-8f63-4fcc-8e4b-69fc23749f37 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:50.886771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:50.915963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76950d30-e32f-4968-ac68-82568cc41be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:50.916627 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76950d30-e32f-4968-ac68-82568cc41be8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:50.916915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76950d30-e32f-4968-ac68-82568cc41be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:50.917196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76950d30-e32f-4968-ac68-82568cc41be8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:50.929086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76950d30-e32f-4968-ac68-82568cc41be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:50.934034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f8b89-3c44-4fc4-b253-8d13837bfc82 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-76950d30-e32f-4968-ac68-82568cc41be8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:50.938584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:50.939537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:50.940476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 172/346] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:50.941160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76950d30-e32f-4968-ac68-82568cc41be8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:50.958459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:50.973951 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e5bba-e235-405f-81c5-e56500ffe02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:50.974612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5bba-e235-405f-81c5-e56500ffe02c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:50.974896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5bba-e235-405f-81c5-e56500ffe02c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:50.975171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5bba-e235-405f-81c5-e56500ffe02c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.048397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e5bba-e235-405f-81c5-e56500ffe02c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:51.049867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76950d30-e32f-4968-ac68-82568cc41be8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:51.050006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e5bba-e235-405f-81c5-e56500ffe02c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:51.089808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc317ffa-fbfb-4e73-816b-68da8a4cfe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:51.090463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc317ffa-fbfb-4e73-816b-68da8a4cfe3a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.090747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc317ffa-fbfb-4e73-816b-68da8a4cfe3a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.091027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc317ffa-fbfb-4e73-816b-68da8a4cfe3a None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.092409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc317ffa-fbfb-4e73-816b-68da8a4cfe3a None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:51.093883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc317ffa-fbfb-4e73-816b-68da8a4cfe3a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:51.095730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 173/347] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:50 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:51.096947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc317ffa-fbfb-4e73-816b-68da8a4cfe3a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:51.113399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:51.158999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 175/348] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:51.159865 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e5bba-e235-405f-81c5-e56500ffe02c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:51.188421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:51.207330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc317ffa-fbfb-4e73-816b-68da8a4cfe3a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:51.243527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fec8b8-ccf2-4b14-a64d-4c04756e6df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:51.245267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fec8b8-ccf2-4b14-a64d-4c04756e6df2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.245553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fec8b8-ccf2-4b14-a64d-4c04756e6df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.245827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fec8b8-ccf2-4b14-a64d-4c04756e6df2 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.246558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fec8b8-ccf2-4b14-a64d-4c04756e6df2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:51.246869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fec8b8-ccf2-4b14-a64d-4c04756e6df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:51.248289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fec8b8-ccf2-4b14-a64d-4c04756e6df2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:51.257564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 174/349] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:51.258918 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9e4e-3394-4695-ac49-0c76be017060 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.259467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9e4e-3394-4695-ac49-0c76be017060 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.259820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe9e4e-3394-4695-ac49-0c76be017060 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.261316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 175/350] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:51.262042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9e4e-3394-4695-ac49-0c76be017060 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:51.266343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e5bba-e235-405f-81c5-e56500ffe02c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:51.310409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6080d1c-5d34-4dc7-8e53-14c2258e0847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:51.311153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6080d1c-5d34-4dc7-8e53-14c2258e0847 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.311507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6080d1c-5d34-4dc7-8e53-14c2258e0847 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.311806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6080d1c-5d34-4dc7-8e53-14c2258e0847 None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.321362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6080d1c-5d34-4dc7-8e53-14c2258e0847 None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:51.322957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6080d1c-5d34-4dc7-8e53-14c2258e0847 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:51.324807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 176/351] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:51 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:51.326246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d94117-09b9-4628-bf0d-ee6a220206bc None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.326568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d94117-09b9-4628-bf0d-ee6a220206bc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.326927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d94117-09b9-4628-bf0d-ee6a220206bc None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.329372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 177/352] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:51.330531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f4ca3-b637-4ad7-9111-6c9ef32a2074 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.330817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f4ca3-b637-4ad7-9111-6c9ef32a2074 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.331090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f4ca3-b637-4ad7-9111-6c9ef32a2074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.350145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a86f4ca3-b637-4ad7-9111-6c9ef32a2074 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:51.351360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a86f4ca3-b637-4ad7-9111-6c9ef32a2074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:51.372826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:51.394566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe9e4e-3394-4695-ac49-0c76be017060 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:51.445763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0220f66-cb94-4f9c-b049-96a2b89b25f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:51.446583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0220f66-cb94-4f9c-b049-96a2b89b25f7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.446863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0220f66-cb94-4f9c-b049-96a2b89b25f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.447136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0220f66-cb94-4f9c-b049-96a2b89b25f7 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.447665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 178/353] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:51.448488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f4ca3-b637-4ad7-9111-6c9ef32a2074 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:51.449509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0220f66-cb94-4f9c-b049-96a2b89b25f7 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:51.452048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0220f66-cb94-4f9c-b049-96a2b89b25f7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:51.454565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 176/354] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:51 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:51.454565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eab56d-d5ba-4a1f-bf0a-734d9d610e2e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.454761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eab56d-d5ba-4a1f-bf0a-734d9d610e2e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.459771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eab56d-d5ba-4a1f-bf0a-734d9d610e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.484158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e9eab56d-d5ba-4a1f-bf0a-734d9d610e2e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:51.485548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e9eab56d-d5ba-4a1f-bf0a-734d9d610e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:51.494728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:51.580387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 177/355] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:51.580757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f4ca3-b637-4ad7-9111-6c9ef32a2074 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:51.581355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eab56d-d5ba-4a1f-bf0a-734d9d610e2e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:51.627200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d9ce7-afd2-40b1-96dc-cfaa3c947564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:51.627861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d9ce7-afd2-40b1-96dc-cfaa3c947564 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.628134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d9ce7-afd2-40b1-96dc-cfaa3c947564 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.628407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d9ce7-afd2-40b1-96dc-cfaa3c947564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.654433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:51.702364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eab56d-d5ba-4a1f-bf0a-734d9d610e2e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:51.713701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d9ce7-afd2-40b1-96dc-cfaa3c947564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:51.715736 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d9ce7-afd2-40b1-96dc-cfaa3c947564 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:51.749402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033d53-a316-46b4-8cc5-1302ca21b5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:51.750041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033d53-a316-46b4-8cc5-1302ca21b5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.750324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033d53-a316-46b4-8cc5-1302ca21b5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.750601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033d53-a316-46b4-8cc5-1302ca21b5e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.751391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033d53-a316-46b4-8cc5-1302ca21b5e8 None admin] RBAC: Adding query filter params (group.id=1e2af7c4177844c3a10e4d4665ecfff8, role.id=512a743b8ca748f8b793681748be7906, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:51.751704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033d53-a316-46b4-8cc5-1302ca21b5e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:51.753651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033d53-a316-46b4-8cc5-1302ca21b5e8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:51.759367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 178/356] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Aug 6 01:04:51 2022] GET /identity/v3/role_assignments?group.id=1e2af7c4177844c3a10e4d4665ecfff8&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=512a743b8ca748f8b793681748be7906 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:51.760446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033d53-a316-46b4-8cc5-1302ca21b5e8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:51.769301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:51.849473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 179/357] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:51.851466 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d9ce7-afd2-40b1-96dc-cfaa3c947564 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:51.874425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033d53-a316-46b4-8cc5-1302ca21b5e8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:51.913312 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975141e-42dc-415a-94af-534ca769ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:51.914248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975141e-42dc-415a-94af-534ca769ff49 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:51.915165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975141e-42dc-415a-94af-534ca769ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:51.915598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975141e-42dc-415a-94af-534ca769ff49 None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:51.917316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975141e-42dc-415a-94af-534ca769ff49 None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:51.918168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:51.919627 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975141e-42dc-415a-94af-534ca769ff49 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:51.922572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 179/358] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:51 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:51.923633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975141e-42dc-415a-94af-534ca769ff49 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:51.980426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d9ce7-afd2-40b1-96dc-cfaa3c947564 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:52.016696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:52.018624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc8648-297b-4c8d-9d5c-d8f6f3e43d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:52.019558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc8648-297b-4c8d-9d5c-d8f6f3e43d9d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.019930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc8648-297b-4c8d-9d5c-d8f6f3e43d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.020213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc8648-297b-4c8d-9d5c-d8f6f3e43d9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.033280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc8648-297b-4c8d-9d5c-d8f6f3e43d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-45bc8648-297b-4c8d-9d5c-d8f6f3e43d9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:52.039338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:52.040365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:52.041174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:52.041802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 180/359] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:52.042445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc8648-297b-4c8d-9d5c-d8f6f3e43d9d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:52.053464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975141e-42dc-415a-94af-534ca769ff49 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:52.100959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23afe5f-2ed0-41a8-9d24-7582d0e9b62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:52.101699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23afe5f-2ed0-41a8-9d24-7582d0e9b62d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.101987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23afe5f-2ed0-41a8-9d24-7582d0e9b62d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.102266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23afe5f-2ed0-41a8-9d24-7582d0e9b62d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.114615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23afe5f-2ed0-41a8-9d24-7582d0e9b62d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e23afe5f-2ed0-41a8-9d24-7582d0e9b62d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:52.120175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:52.121064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:52.122266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 180/360] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:52.122935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff572fe-5886-4ddb-b462-6ee8a3e6614c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.123238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff572fe-5886-4ddb-b462-6ee8a3e6614c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.123386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff572fe-5886-4ddb-b462-6ee8a3e6614c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.124720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 181/361] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:52.125840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff572fe-5886-4ddb-b462-6ee8a3e6614c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:52.167020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc8648-297b-4c8d-9d5c-d8f6f3e43d9d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:52.182665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:52.209645 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf842b6-b34b-4854-b24b-31fb45c99f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:52.210298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf842b6-b34b-4854-b24b-31fb45c99f86 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.210598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf842b6-b34b-4854-b24b-31fb45c99f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.210857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf842b6-b34b-4854-b24b-31fb45c99f86 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.215679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf842b6-b34b-4854-b24b-31fb45c99f86 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:52.215679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf842b6-b34b-4854-b24b-31fb45c99f86 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:52.215679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 181/362] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:52 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:52.217588 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf842b6-b34b-4854-b24b-31fb45c99f86 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:52.266791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff572fe-5886-4ddb-b462-6ee8a3e6614c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:52.315059 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4977e29-7edf-4761-8de5-5938fef14e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:52.315874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4977e29-7edf-4761-8de5-5938fef14e11 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.316162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4977e29-7edf-4761-8de5-5938fef14e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.316440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4977e29-7edf-4761-8de5-5938fef14e11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.317235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4977e29-7edf-4761-8de5-5938fef14e11 None admin] RBAC: Adding query filter params (group.id=1e2af7c4177844c3a10e4d4665ecfff8, role.id=b5b241afd53c456e9a9fa3033eea74ad, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:52.317549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4977e29-7edf-4761-8de5-5938fef14e11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:52.319908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4977e29-7edf-4761-8de5-5938fef14e11 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:52.321890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:52.326858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 182/363] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Aug 6 01:04:52 2022] GET /identity/v3/role_assignments?group.id=1e2af7c4177844c3a10e4d4665ecfff8&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=b5b241afd53c456e9a9fa3033eea74ad => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:52.327653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4977e29-7edf-4761-8de5-5938fef14e11 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:52.349910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf842b6-b34b-4854-b24b-31fb45c99f86 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:52.406668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328fd62-af79-436e-8d4b-92af9a0793fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:52.407628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328fd62-af79-436e-8d4b-92af9a0793fa None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.407967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328fd62-af79-436e-8d4b-92af9a0793fa None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.408244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328fd62-af79-436e-8d4b-92af9a0793fa None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.409260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328fd62-af79-436e-8d4b-92af9a0793fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:52.409570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328fd62-af79-436e-8d4b-92af9a0793fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:52.410969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328fd62-af79-436e-8d4b-92af9a0793fa None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:52.423129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 182/364] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:52.424971 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8351d-89b3-4e41-a11b-23950b255cbd None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.425261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8351d-89b3-4e41-a11b-23950b255cbd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.425538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8351d-89b3-4e41-a11b-23950b255cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.449557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4ee8351d-89b3-4e41-a11b-23950b255cbd None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:52.451314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4ee8351d-89b3-4e41-a11b-23950b255cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:52.466027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4977e29-7edf-4761-8de5-5938fef14e11 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:52.509128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:52.512516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d4387-3302-4d09-80d7-a20d631b6cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:52.513148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d4387-3302-4d09-80d7-a20d631b6cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.513428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d4387-3302-4d09-80d7-a20d631b6cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.513703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d4387-3302-4d09-80d7-a20d631b6cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.514427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d4387-3302-4d09-80d7-a20d631b6cd5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:52.514742 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d4387-3302-4d09-80d7-a20d631b6cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:52.525842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d4387-3302-4d09-80d7-a20d631b6cd5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:52.528788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 183/365] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:52.530024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d4387-3302-4d09-80d7-a20d631b6cd5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:52.542164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 183/366] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:52.543428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8351d-89b3-4e41-a11b-23950b255cbd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:52.613481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:52.667633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d4387-3302-4d09-80d7-a20d631b6cd5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:52.679394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8351d-89b3-4e41-a11b-23950b255cbd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:52.719390 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396f471-9a29-44aa-be72-9ac29e5391e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:52.723519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396f471-9a29-44aa-be72-9ac29e5391e8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.723519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396f471-9a29-44aa-be72-9ac29e5391e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.723519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396f471-9a29-44aa-be72-9ac29e5391e8 None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.723519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396f471-9a29-44aa-be72-9ac29e5391e8 None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:52.725100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d396f471-9a29-44aa-be72-9ac29e5391e8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:52.726875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 184/367] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:52 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:52.727474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396f471-9a29-44aa-be72-9ac29e5391e8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:52.737029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80226504-52f6-481f-80a1-312d01028096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:52.737649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80226504-52f6-481f-80a1-312d01028096 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.737929 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80226504-52f6-481f-80a1-312d01028096 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.738210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80226504-52f6-481f-80a1-312d01028096 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.805318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:52.843925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80226504-52f6-481f-80a1-312d01028096 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:52.846100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80226504-52f6-481f-80a1-312d01028096 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:52.859311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396f471-9a29-44aa-be72-9ac29e5391e8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:52.889065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:52.898002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe53e0e-e023-47cf-84a2-a412b6ff8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:52.898748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe53e0e-e023-47cf-84a2-a412b6ff8adc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:52.899191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe53e0e-e023-47cf-84a2-a412b6ff8adc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:52.899440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe53e0e-e023-47cf-84a2-a412b6ff8adc None admin] PATH_INFO: `/v3/users/e67adddace0844b6b0530f3c4a869a64` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:52.900756 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe53e0e-e023-47cf-84a2-a412b6ff8adc None admin] RBAC: Authorizing `identity:get_user(user_id=e67adddace0844b6b0530f3c4a869a64)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:52.902645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe53e0e-e023-47cf-84a2-a412b6ff8adc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:52.904577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 185/368] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:04:52 2022] GET /identity/v3/users/e67adddace0844b6b0530f3c4a869a64 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:52.905541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe53e0e-e023-47cf-84a2-a412b6ff8adc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:52.963517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 184/369] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:52.964877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80226504-52f6-481f-80a1-312d01028096 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:53.019375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe53e0e-e023-47cf-84a2-a412b6ff8adc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:53.050129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:53.061243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece5f05-adf5-46de-b62c-8bf60fc983ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:53.061243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece5f05-adf5-46de-b62c-8bf60fc983ad None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:53.061243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece5f05-adf5-46de-b62c-8bf60fc983ad None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:53.061243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece5f05-adf5-46de-b62c-8bf60fc983ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:53.070216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece5f05-adf5-46de-b62c-8bf60fc983ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-fece5f05-adf5-46de-b62c-8bf60fc983ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:53.076708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:53.077620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:53.078478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 186/370] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:53.084137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece5f05-adf5-46de-b62c-8bf60fc983ad None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:53.087757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80226504-52f6-481f-80a1-312d01028096 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:53.133232 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00b991-6d69-4abd-8b87-f843d69414d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:53.134720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00b991-6d69-4abd-8b87-f843d69414d2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:53.135392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00b991-6d69-4abd-8b87-f843d69414d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:53.136184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00b991-6d69-4abd-8b87-f843d69414d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:53.137740 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00b991-6d69-4abd-8b87-f843d69414d2 None admin] RBAC: Adding query filter params (group.id=1e2af7c4177844c3a10e4d4665ecfff8, role.id=512a743b8ca748f8b793681748be7906, scope.project.id=3059810e56d143ab91728e2220010d53) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:53.138585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00b991-6d69-4abd-8b87-f843d69414d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:53.141204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00b991-6d69-4abd-8b87-f843d69414d2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:53.149360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:53.152902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 185/371] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Aug 6 01:04:52 2022] GET /identity/v3/role_assignments?group.id=1e2af7c4177844c3a10e4d4665ecfff8&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=512a743b8ca748f8b793681748be7906 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:53.154170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00b991-6d69-4abd-8b87-f843d69414d2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:53.208401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece5f05-adf5-46de-b62c-8bf60fc983ad None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:53.252930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0910f1d-75f2-4fa4-9c0b-cd0fb881d05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:53.253586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0910f1d-75f2-4fa4-9c0b-cd0fb881d05d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:53.253875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0910f1d-75f2-4fa4-9c0b-cd0fb881d05d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:53.254154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0910f1d-75f2-4fa4-9c0b-cd0fb881d05d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:53.254976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0910f1d-75f2-4fa4-9c0b-cd0fb881d05d None admin] RBAC: Adding query filter params (role.id=324a803dd87d4027b73d871fe46ea228, user.id=e67adddace0844b6b0530f3c4a869a64, scope.system=all) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:53.255295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0910f1d-75f2-4fa4-9c0b-cd0fb881d05d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:53.256903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0910f1d-75f2-4fa4-9c0b-cd0fb881d05d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:53.281756 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 187/372] 10.222.0.25 () {56 vars in 1308 bytes} [Sat Aug 6 01:04:53 2022] GET /identity/v3/role_assignments?user.id=e67adddace0844b6b0530f3c4a869a64&scope.system=all&role.id=324a803dd87d4027b73d871fe46ea228 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:53.282626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0910f1d-75f2-4fa4-9c0b-cd0fb881d05d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:53.306009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00b991-6d69-4abd-8b87-f843d69414d2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:53.347578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8415c2f-9613-435e-9e63-0fb9e501b19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:53.348627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415c2f-9613-435e-9e63-0fb9e501b19b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:53.349155 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415c2f-9613-435e-9e63-0fb9e501b19b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:53.350241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8415c2f-9613-435e-9e63-0fb9e501b19b None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:53.351753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8415c2f-9613-435e-9e63-0fb9e501b19b None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:53.353514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8415c2f-9613-435e-9e63-0fb9e501b19b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:53.355829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 186/373] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:53 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:53.361818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8415c2f-9613-435e-9e63-0fb9e501b19b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:53.372449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:53.456726 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0910f1d-75f2-4fa4-9c0b-cd0fb881d05d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:53.481785 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:53.523201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d736b7-098d-42c8-97ad-cb09b7a86ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:53.525888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d736b7-098d-42c8-97ad-cb09b7a86ada None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:53.526637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d736b7-098d-42c8-97ad-cb09b7a86ada None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:53.528216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d736b7-098d-42c8-97ad-cb09b7a86ada None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:53.529003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d736b7-098d-42c8-97ad-cb09b7a86ada None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:53.529324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d736b7-098d-42c8-97ad-cb09b7a86ada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:53.530775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d736b7-098d-42c8-97ad-cb09b7a86ada None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:53.543407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 188/374] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:53.549680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d736b7-098d-42c8-97ad-cb09b7a86ada None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:53.558003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8415c2f-9613-435e-9e63-0fb9e501b19b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:53.635401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10cd21-f7ac-4fc9-9702-f2a485ca1df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:53.640037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10cd21-f7ac-4fc9-9702-f2a485ca1df3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:53.640597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10cd21-f7ac-4fc9-9702-f2a485ca1df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:53.641069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10cd21-f7ac-4fc9-9702-f2a485ca1df3 None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:53.642486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10cd21-f7ac-4fc9-9702-f2a485ca1df3 None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:53.644270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10cd21-f7ac-4fc9-9702-f2a485ca1df3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:53.647038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 187/375] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:53 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:53.669811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10cd21-f7ac-4fc9-9702-f2a485ca1df3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:53.676289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d736b7-098d-42c8-97ad-cb09b7a86ada None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:53.715486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ab4f-c510-4fe9-916a-539302557b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:53.716466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ab4f-c510-4fe9-916a-539302557b3b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:53.717923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ab4f-c510-4fe9-916a-539302557b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:53.717923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ab4f-c510-4fe9-916a-539302557b3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:53.769969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:53.796426 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ab4f-c510-4fe9-916a-539302557b3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:53.798275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ab4f-c510-4fe9-916a-539302557b3b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:53.839074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:53.888213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10cd21-f7ac-4fc9-9702-f2a485ca1df3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:53.914471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 189/376] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:53.920147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ab4f-c510-4fe9-916a-539302557b3b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:53.935296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76225c9b-4247-4f18-b710-d929b0db511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:53.936335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76225c9b-4247-4f18-b710-d929b0db511b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:53.936827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76225c9b-4247-4f18-b710-d929b0db511b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:53.937340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76225c9b-4247-4f18-b710-d929b0db511b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:53.938374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76225c9b-4247-4f18-b710-d929b0db511b None admin] RBAC: Adding query filter params (group.id=1e2af7c4177844c3a10e4d4665ecfff8, role.id=b5b241afd53c456e9a9fa3033eea74ad, scope.project.id=3059810e56d143ab91728e2220010d53) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:53.938949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76225c9b-4247-4f18-b710-d929b0db511b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:53.942808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76225c9b-4247-4f18-b710-d929b0db511b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:53.953326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 188/377] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Aug 6 01:04:53 2022] GET /identity/v3/role_assignments?group.id=1e2af7c4177844c3a10e4d4665ecfff8&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=b5b241afd53c456e9a9fa3033eea74ad => generated 260 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:54.013121 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:54.039681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ab4f-c510-4fe9-916a-539302557b3b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:54.088729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cf520-3058-450a-a60a-bc1bb7ae5eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:54.090337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cf520-3058-450a-a60a-bc1bb7ae5eac None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:54.090963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cf520-3058-450a-a60a-bc1bb7ae5eac None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:54.091506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cf520-3058-450a-a60a-bc1bb7ae5eac None admin] PATH_INFO: `/v3/projects/58e81a1daacf4bb08ea62cf133cf1073` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:54.093153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cf520-3058-450a-a60a-bc1bb7ae5eac None admin] RBAC: Authorizing `identity:get_project(project_id=58e81a1daacf4bb08ea62cf133cf1073)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:54.094887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cf520-3058-450a-a60a-bc1bb7ae5eac None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:54.097332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 190/378] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:53 2022] GET /identity/v3/projects/58e81a1daacf4bb08ea62cf133cf1073 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:54.102690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76225c9b-4247-4f18-b710-d929b0db511b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:54.233614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76225c9b-4247-4f18-b710-d929b0db511b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:54.279428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b438f7-89c8-461a-b467-1f8d14074a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:54.280129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b438f7-89c8-461a-b467-1f8d14074a6d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:54.280450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b438f7-89c8-461a-b467-1f8d14074a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:54.280736 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b438f7-89c8-461a-b467-1f8d14074a6d None admin] PATH_INFO: `/v3/groups/0d9aa4e1620a48fe9fd7e8dca2865e04` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:54.288536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b438f7-89c8-461a-b467-1f8d14074a6d None admin] RBAC: Authorizing `identity:get_group(group_id=0d9aa4e1620a48fe9fd7e8dca2865e04)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:54.290002 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b438f7-89c8-461a-b467-1f8d14074a6d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:54.291893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 189/379] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:54 2022] GET /identity/v3/groups/0d9aa4e1620a48fe9fd7e8dca2865e04 => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:54.297660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cf520-3058-450a-a60a-bc1bb7ae5eac None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:54.314186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902f9d8-87f4-4653-9420-30ef4aacb596 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:54.314186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902f9d8-87f4-4653-9420-30ef4aacb596 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:54.314461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902f9d8-87f4-4653-9420-30ef4aacb596 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:54.317903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 190/380] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:54.327525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87036822-8aaf-4a37-b8a9-81991edc041e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:54.327922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87036822-8aaf-4a37-b8a9-81991edc041e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:54.328210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87036822-8aaf-4a37-b8a9-81991edc041e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:54.341325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:54.357994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-87036822-8aaf-4a37-b8a9-81991edc041e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:54.360521 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-87036822-8aaf-4a37-b8a9-81991edc041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:54.489390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 191/381] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:54.495678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cf520-3058-450a-a60a-bc1bb7ae5eac None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:54.538392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53075ac-3be9-4989-b683-2cba34c65670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:54.539076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53075ac-3be9-4989-b683-2cba34c65670 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:54.539365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53075ac-3be9-4989-b683-2cba34c65670 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:54.539643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53075ac-3be9-4989-b683-2cba34c65670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:54.540482 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53075ac-3be9-4989-b683-2cba34c65670 None admin] RBAC: Adding query filter params (group.id=0d9aa4e1620a48fe9fd7e8dca2865e04, role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=58e81a1daacf4bb08ea62cf133cf1073) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:54.540794 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53075ac-3be9-4989-b683-2cba34c65670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:54.542256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53075ac-3be9-4989-b683-2cba34c65670 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:54.551231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 191/382] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Aug 6 01:04:54 2022] GET /identity/v3/role_assignments?group.id=0d9aa4e1620a48fe9fd7e8dca2865e04&scope.project.id=58e81a1daacf4bb08ea62cf133cf1073&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:54.575517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87036822-8aaf-4a37-b8a9-81991edc041e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:54.712621 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87036822-8aaf-4a37-b8a9-81991edc041e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:54.746078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:54.755409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df42b0-fc18-41dc-8f36-ddaf235a26a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:54.756689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df42b0-fc18-41dc-8f36-ddaf235a26a5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:54.756975 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df42b0-fc18-41dc-8f36-ddaf235a26a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:54.757254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df42b0-fc18-41dc-8f36-ddaf235a26a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:54.773579 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df42b0-fc18-41dc-8f36-ddaf235a26a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-24df42b0-fc18-41dc-8f36-ddaf235a26a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:54.778783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:54.779744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:54.780609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 192/383] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:54.786422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53075ac-3be9-4989-b683-2cba34c65670 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:54.810856 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:54.929402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53075ac-3be9-4989-b683-2cba34c65670 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:54.940935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef362b66-95d9-4e60-aebd-0ffb282ee164 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:54.941425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef362b66-95d9-4e60-aebd-0ffb282ee164 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:54.941530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef362b66-95d9-4e60-aebd-0ffb282ee164 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:54.942819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 193/384] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:54.948748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46cdf0-1428-41fb-9210-900e309fe50b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:54.949033 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46cdf0-1428-41fb-9210-900e309fe50b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:54.949310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46cdf0-1428-41fb-9210-900e309fe50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:54.970216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2d46cdf0-1428-41fb-9210-900e309fe50b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:54.971401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2d46cdf0-1428-41fb-9210-900e309fe50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:54.986231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6a6a6-e6b9-4890-9ce4-3f5ac14b336b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:54.986878 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6a6a6-e6b9-4890-9ce4-3f5ac14b336b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:54.987406 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6a6a6-e6b9-4890-9ce4-3f5ac14b336b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:54.987688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6a6a6-e6b9-4890-9ce4-3f5ac14b336b None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:54.989053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6a6a6-e6b9-4890-9ce4-3f5ac14b336b None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:54.989273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6a6a6-e6b9-4890-9ce4-3f5ac14b336b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:54.991219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6a6a6-e6b9-4890-9ce4-3f5ac14b336b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:55.011166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 192/385] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:55.017088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6a6a6-e6b9-4890-9ce4-3f5ac14b336b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:55.087453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 194/386] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:55.113385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46cdf0-1428-41fb-9210-900e309fe50b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:55.128568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:55.162428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6a6a6-e6b9-4890-9ce4-3f5ac14b336b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:55.202764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666b288-dab2-4510-8966-593cd6c9650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:55.203811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666b288-dab2-4510-8966-593cd6c9650f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:55.204303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666b288-dab2-4510-8966-593cd6c9650f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:55.204773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666b288-dab2-4510-8966-593cd6c9650f None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:55.206208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666b288-dab2-4510-8966-593cd6c9650f None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:55.207995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666b288-dab2-4510-8966-593cd6c9650f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:55.210961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 193/387] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:55 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:55.216065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666b288-dab2-4510-8966-593cd6c9650f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:55.315811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46cdf0-1428-41fb-9210-900e309fe50b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:55.328074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666b288-dab2-4510-8966-593cd6c9650f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:55.352197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:55.365379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2949cb-1b1c-4643-8cff-630d990a9f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:55.366614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2949cb-1b1c-4643-8cff-630d990a9f6b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:55.367361 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2949cb-1b1c-4643-8cff-630d990a9f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:55.368308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2949cb-1b1c-4643-8cff-630d990a9f6b None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:55.369055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d296-4ea4-4d15-8450-d30a189ed48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:55.374558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d296-4ea4-4d15-8450-d30a189ed48a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:55.374558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d296-4ea4-4d15-8450-d30a189ed48a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:55.374558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d296-4ea4-4d15-8450-d30a189ed48a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:55.374789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2949cb-1b1c-4643-8cff-630d990a9f6b None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:55.374789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2949cb-1b1c-4643-8cff-630d990a9f6b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:55.375880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 194/388] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:55 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:55.384541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2949cb-1b1c-4643-8cff-630d990a9f6b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:55.391639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1d296-4ea4-4d15-8450-d30a189ed48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:55.398676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-d4f1d296-4ea4-4d15-8450-d30a189ed48a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:55.399699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:55.400614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:55.401418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:55.406282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 195/389] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:55.410542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d296-4ea4-4d15-8450-d30a189ed48a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:55.532175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d296-4ea4-4d15-8450-d30a189ed48a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:55.573093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93167d-a1ab-46f2-8150-16243ba30196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:55.574005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93167d-a1ab-46f2-8150-16243ba30196 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:55.574829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93167d-a1ab-46f2-8150-16243ba30196 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:55.575419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93167d-a1ab-46f2-8150-16243ba30196 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:55.576733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93167d-a1ab-46f2-8150-16243ba30196 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:55.577453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93167d-a1ab-46f2-8150-16243ba30196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:55.579495 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93167d-a1ab-46f2-8150-16243ba30196 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:55.593984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2949cb-1b1c-4643-8cff-630d990a9f6b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:55.594486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 196/390] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:55.597113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93167d-a1ab-46f2-8150-16243ba30196 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:55.650096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:55.697402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e9c91-da45-43c1-8d73-8c27e86c7fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:55.698209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e9c91-da45-43c1-8d73-8c27e86c7fb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:55.698576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e9c91-da45-43c1-8d73-8c27e86c7fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:55.698945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e9c91-da45-43c1-8d73-8c27e86c7fb4 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/groups/1e2af7c4177844c3a10e4d4665ecfff8/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:55.705181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e9c91-da45-43c1-8d73-8c27e86c7fb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fa1f9bede2243f0ba30c5f573dc695b, group_id=1e2af7c4177844c3a10e4d4665ecfff8, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:55.707036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e9c91-da45-43c1-8d73-8c27e86c7fb4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:55.710204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93167d-a1ab-46f2-8150-16243ba30196 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:55.747020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ffb2a-31ba-4c15-adfc-c605704f0d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:55.747991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ffb2a-31ba-4c15-adfc-c605704f0d5a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:55.748472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ffb2a-31ba-4c15-adfc-c605704f0d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:55.748919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ffb2a-31ba-4c15-adfc-c605704f0d5a None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:55.750344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ffb2a-31ba-4c15-adfc-c605704f0d5a None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:55.751992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ffb2a-31ba-4c15-adfc-c605704f0d5a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:55.759015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 197/391] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:55 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:55.760535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ffb2a-31ba-4c15-adfc-c605704f0d5a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:55.784210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 195/392] 10.222.0.25 () {58 vars in 1442 bytes} [Sat Aug 6 01:04:55 2022] PUT /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/groups/1e2af7c4177844c3a10e4d4665ecfff8/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:55.877364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ffb2a-31ba-4c15-adfc-c605704f0d5a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:55.891373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:55.915011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d23003-7b1d-4c1b-8039-d55e913166c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:55.916003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d23003-7b1d-4c1b-8039-d55e913166c1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:55.916487 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d23003-7b1d-4c1b-8039-d55e913166c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:55.916936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d23003-7b1d-4c1b-8039-d55e913166c1 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:55.918400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d23003-7b1d-4c1b-8039-d55e913166c1 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:55.920179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d23003-7b1d-4c1b-8039-d55e913166c1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:55.927374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 198/393] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:55 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:55.929748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e9c91-da45-43c1-8d73-8c27e86c7fb4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:56.015516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:56.058826 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e9c91-da45-43c1-8d73-8c27e86c7fb4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:56.096930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a4042-2b8b-4297-bb84-9f0080810448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:56.097622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a4042-2b8b-4297-bb84-9f0080810448 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:56.097907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a4042-2b8b-4297-bb84-9f0080810448 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:56.098186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a4042-2b8b-4297-bb84-9f0080810448 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/groups/1e2af7c4177844c3a10e4d4665ecfff8/roles/b5b241afd53c456e9a9fa3033eea74ad` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:56.100978 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a4042-2b8b-4297-bb84-9f0080810448 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fa1f9bede2243f0ba30c5f573dc695b, group_id=1e2af7c4177844c3a10e4d4665ecfff8, role_id=b5b241afd53c456e9a9fa3033eea74ad)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:56.102463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a4042-2b8b-4297-bb84-9f0080810448 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:56.173618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 196/394] 10.222.0.25 () {58 vars in 1442 bytes} [Sat Aug 6 01:04:55 2022] PUT /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b/groups/1e2af7c4177844c3a10e4d4665ecfff8/roles/b5b241afd53c456e9a9fa3033eea74ad => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:56.185978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa162749-f382-481b-99dc-553b03271ff0 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:56.186476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa162749-f382-481b-99dc-553b03271ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:56.186548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa162749-f382-481b-99dc-553b03271ff0 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:56.188154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 199/395] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:56.194045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751d23b-c515-4905-9dd0-57ccc2374b62 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:56.194346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751d23b-c515-4905-9dd0-57ccc2374b62 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:56.194617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751d23b-c515-4905-9dd0-57ccc2374b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:56.217844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5751d23b-c515-4905-9dd0-57ccc2374b62 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:56.219299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5751d23b-c515-4905-9dd0-57ccc2374b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:56.305543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 197/396] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:56.343597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa162749-f382-481b-99dc-553b03271ff0 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:56.465881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa162749-f382-481b-99dc-553b03271ff0 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:56.500360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e15bf7-96a0-48ad-ae2a-c1d2cab867de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:56.501335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e15bf7-96a0-48ad-ae2a-c1d2cab867de None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:56.501810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e15bf7-96a0-48ad-ae2a-c1d2cab867de None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:56.502298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e15bf7-96a0-48ad-ae2a-c1d2cab867de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:56.512999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e15bf7-96a0-48ad-ae2a-c1d2cab867de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-56e15bf7-96a0-48ad-ae2a-c1d2cab867de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:56.518535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:56.519417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:56.520236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:56.523373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 200/397] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:56.528815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751d23b-c515-4905-9dd0-57ccc2374b62 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:56.596848 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:56.602462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f819a6-4049-478b-a31f-13387c689e3b None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:56.602889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f819a6-4049-478b-a31f-13387c689e3b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:56.603167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f819a6-4049-478b-a31f-13387c689e3b None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:56.605073 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 201/398] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:56.610642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fd6a6-a15f-4ce2-8357-4576f73fc71f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:56.610933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fd6a6-a15f-4ce2-8357-4576f73fc71f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:56.611204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fd6a6-a15f-4ce2-8357-4576f73fc71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:56.632314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1c8fd6a6-a15f-4ce2-8357-4576f73fc71f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:56.633522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1c8fd6a6-a15f-4ce2-8357-4576f73fc71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:56.681916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:56.702143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751d23b-c515-4905-9dd0-57ccc2374b62 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:56.710325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 202/399] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:56.754126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb960fff-1aa4-41dc-8869-1c775c0f3317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:56.754675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb960fff-1aa4-41dc-8869-1c775c0f3317 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:56.754954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb960fff-1aa4-41dc-8869-1c775c0f3317 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:56.755229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb960fff-1aa4-41dc-8869-1c775c0f3317 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:56.756513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb960fff-1aa4-41dc-8869-1c775c0f3317 None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:56.756818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb960fff-1aa4-41dc-8869-1c775c0f3317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:56.758201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb960fff-1aa4-41dc-8869-1c775c0f3317 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:56.766744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 198/400] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:56.770717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fd6a6-a15f-4ce2-8357-4576f73fc71f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:56.781833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb960fff-1aa4-41dc-8869-1c775c0f3317 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:56.920695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb960fff-1aa4-41dc-8869-1c775c0f3317 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:56.940243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:56.960072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887ecb7-a0da-4f90-a7df-f48cf82b7219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:56.960759 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887ecb7-a0da-4f90-a7df-f48cf82b7219 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:56.961035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887ecb7-a0da-4f90-a7df-f48cf82b7219 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:56.961316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887ecb7-a0da-4f90-a7df-f48cf82b7219 None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:56.962583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887ecb7-a0da-4f90-a7df-f48cf82b7219 None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:56.964079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887ecb7-a0da-4f90-a7df-f48cf82b7219 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:56.964261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fd6a6-a15f-4ce2-8357-4576f73fc71f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:56.965999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 199/401] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:56 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:56.971388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887ecb7-a0da-4f90-a7df-f48cf82b7219 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:57.032057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc615a-2864-40cb-9409-a676831aa08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:57.032619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc615a-2864-40cb-9409-a676831aa08b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:57.032903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc615a-2864-40cb-9409-a676831aa08b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:57.033179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc615a-2864-40cb-9409-a676831aa08b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:57.054447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc615a-2864-40cb-9409-a676831aa08b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b5cc615a-2864-40cb-9409-a676831aa08b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:57.067911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:57.069371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:57.070164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:57.072818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 203/402] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:57.078648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc615a-2864-40cb-9409-a676831aa08b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:57.109895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887ecb7-a0da-4f90-a7df-f48cf82b7219 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:57.116706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:57.146191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef29d2c-5880-47cc-aafe-eb34e0b8ff13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:57.146908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef29d2c-5880-47cc-aafe-eb34e0b8ff13 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:57.147168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef29d2c-5880-47cc-aafe-eb34e0b8ff13 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:57.147466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef29d2c-5880-47cc-aafe-eb34e0b8ff13 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:57.148778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef29d2c-5880-47cc-aafe-eb34e0b8ff13 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:57.150310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef29d2c-5880-47cc-aafe-eb34e0b8ff13 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:57.152357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 200/403] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:56 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:57.158161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef29d2c-5880-47cc-aafe-eb34e0b8ff13 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:57.195078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc615a-2864-40cb-9409-a676831aa08b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:57.216556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:57.240798 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a966481-64f1-435f-bbf6-480659711980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:57.241805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a966481-64f1-435f-bbf6-480659711980 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:57.242303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a966481-64f1-435f-bbf6-480659711980 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:57.242768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a966481-64f1-435f-bbf6-480659711980 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:57.243681 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a966481-64f1-435f-bbf6-480659711980 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:57.247883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a966481-64f1-435f-bbf6-480659711980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:57.249502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a966481-64f1-435f-bbf6-480659711980 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:57.266593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 204/404] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:57.269495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef29d2c-5880-47cc-aafe-eb34e0b8ff13 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:57.271980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a966481-64f1-435f-bbf6-480659711980 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:57.313542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16a0ab-48a3-4fe7-b74e-734a48632948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:57.314158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a0ab-48a3-4fe7-b74e-734a48632948 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:57.314443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a0ab-48a3-4fe7-b74e-734a48632948 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:57.314722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a0ab-48a3-4fe7-b74e-734a48632948 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53/groups/1e2af7c4177844c3a10e4d4665ecfff8/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:57.322362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16a0ab-48a3-4fe7-b74e-734a48632948 None admin] RBAC: Authorizing `identity:create_grant(project_id=3059810e56d143ab91728e2220010d53, group_id=1e2af7c4177844c3a10e4d4665ecfff8, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:57.323941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16a0ab-48a3-4fe7-b74e-734a48632948 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:57.376457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:57.385992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a966481-64f1-435f-bbf6-480659711980 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:57.431876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 201/405] 10.222.0.25 () {58 vars in 1442 bytes} [Sat Aug 6 01:04:57 2022] PUT /identity/v3/projects/3059810e56d143ab91728e2220010d53/groups/1e2af7c4177844c3a10e4d4665ecfff8/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:57.433745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107175e-561b-428f-89f4-2f994825c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:57.434458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107175e-561b-428f-89f4-2f994825c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:57.434739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107175e-561b-428f-89f4-2f994825c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:57.435022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107175e-561b-428f-89f4-2f994825c1c6 None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:57.441164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107175e-561b-428f-89f4-2f994825c1c6 None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:57.442847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107175e-561b-428f-89f4-2f994825c1c6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:57.443307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a2f40-8549-45fc-b530-bac943fd4161 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:57.443840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a2f40-8549-45fc-b530-bac943fd4161 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:57.444351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a2f40-8549-45fc-b530-bac943fd4161 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:57.445029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 205/406] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:57 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:57.446161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 202/407] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:57.451557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107175e-561b-428f-89f4-2f994825c1c6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:57.469541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5ab8-827a-484c-b48a-b253c8cfb0cb None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:57.469855 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5ab8-827a-484c-b48a-b253c8cfb0cb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:57.470091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5ab8-827a-484c-b48a-b253c8cfb0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:57.513657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-058e5ab8-827a-484c-b48a-b253c8cfb0cb None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:57.515801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-058e5ab8-827a-484c-b48a-b253c8cfb0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:57.616259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107175e-561b-428f-89f4-2f994825c1c6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:57.634075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 203/408] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:57.656859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5ab8-827a-484c-b48a-b253c8cfb0cb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:57.657108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647bd08-0f1f-45e6-81e8-4604cbe2ca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:57.657886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647bd08-0f1f-45e6-81e8-4604cbe2ca4f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:57.658242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647bd08-0f1f-45e6-81e8-4604cbe2ca4f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:57.658596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647bd08-0f1f-45e6-81e8-4604cbe2ca4f None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:57.659939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647bd08-0f1f-45e6-81e8-4604cbe2ca4f None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:57.661476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647bd08-0f1f-45e6-81e8-4604cbe2ca4f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:57.663526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 206/409] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:57 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:57.672890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647bd08-0f1f-45e6-81e8-4604cbe2ca4f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:57.713738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:57.724055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:57.784559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5ab8-827a-484c-b48a-b253c8cfb0cb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:57.816744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647bd08-0f1f-45e6-81e8-4604cbe2ca4f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:57.827981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c0a0e-11e2-4259-96c4-763d90e68b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:57.828668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c0a0e-11e2-4259-96c4-763d90e68b1b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:57.829385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c0a0e-11e2-4259-96c4-763d90e68b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:57.829670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c0a0e-11e2-4259-96c4-763d90e68b1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:57.845362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c0a0e-11e2-4259-96c4-763d90e68b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-008c0a0e-11e2-4259-96c4-763d90e68b1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:57.852622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:57.853492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:57.854305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 204/410] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:04:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:57.859253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fd224-6131-4f84-883b-ba219a42fafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:57.864971 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fd224-6131-4f84-883b-ba219a42fafc None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:57.864971 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fd224-6131-4f84-883b-ba219a42fafc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:57.864971 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fd224-6131-4f84-883b-ba219a42fafc None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53/groups/1e2af7c4177844c3a10e4d4665ecfff8/roles/b5b241afd53c456e9a9fa3033eea74ad` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:57.865725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c0a0e-11e2-4259-96c4-763d90e68b1b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:57.865796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fd224-6131-4f84-883b-ba219a42fafc None admin] RBAC: Authorizing `identity:create_grant(project_id=3059810e56d143ab91728e2220010d53, group_id=1e2af7c4177844c3a10e4d4665ecfff8, role_id=b5b241afd53c456e9a9fa3033eea74ad)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:57.865796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fd224-6131-4f84-883b-ba219a42fafc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:57.953316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 207/411] 10.222.0.25 () {58 vars in 1442 bytes} [Sat Aug 6 01:04:57 2022] PUT /identity/v3/projects/3059810e56d143ab91728e2220010d53/groups/1e2af7c4177844c3a10e4d4665ecfff8/roles/b5b241afd53c456e9a9fa3033eea74ad => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:04:57.973053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c0a0e-11e2-4259-96c4-763d90e68b1b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:57.975947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:58.010682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc0bd-6991-433a-a998-cce668f543e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:58.011324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc0bd-6991-433a-a998-cce668f543e1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:58.011612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc0bd-6991-433a-a998-cce668f543e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:58.011916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc0bd-6991-433a-a998-cce668f543e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:58.012649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cc0bd-6991-433a-a998-cce668f543e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:58.012962 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cc0bd-6991-433a-a998-cce668f543e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:58.014370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cc0bd-6991-433a-a998-cce668f543e1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:58.030424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 205/412] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:58.036251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fd224-6131-4f84-883b-ba219a42fafc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:58.147328 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:58.159795 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fd224-6131-4f84-883b-ba219a42fafc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:58.197419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d00688-2c29-44da-ac35-bc2de4d3e57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:58.198122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d00688-2c29-44da-ac35-bc2de4d3e57f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:58.198349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d00688-2c29-44da-ac35-bc2de4d3e57f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:58.198599 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d00688-2c29-44da-ac35-bc2de4d3e57f None admin] PATH_INFO: `/v3/groups/0d9aa4e1620a48fe9fd7e8dca2865e04` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:58.199862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d00688-2c29-44da-ac35-bc2de4d3e57f None admin] RBAC: Authorizing `identity:get_group(group_id=0d9aa4e1620a48fe9fd7e8dca2865e04)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:58.201679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d00688-2c29-44da-ac35-bc2de4d3e57f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:58.207888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 208/413] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:58 2022] GET /identity/v3/groups/0d9aa4e1620a48fe9fd7e8dca2865e04 => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:58.209292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc0bd-6991-433a-a998-cce668f543e1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:58.324008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc0bd-6991-433a-a998-cce668f543e1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:58.343515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4182635-5bf7-4393-8b46-463ef077a380 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:58.343823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4182635-5bf7-4393-8b46-463ef077a380 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:58.344099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4182635-5bf7-4393-8b46-463ef077a380 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:58.345451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 209/414] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:04:58.350983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342102f-4c7f-49ba-8a52-e008deebefd1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:58.351275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342102f-4c7f-49ba-8a52-e008deebefd1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:58.351554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342102f-4c7f-49ba-8a52-e008deebefd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:58.367370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94d500-2153-4343-bade-a7a9cbae90a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:58.368063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94d500-2153-4343-bade-a7a9cbae90a9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:58.368346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94d500-2153-4343-bade-a7a9cbae90a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:58.368680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94d500-2153-4343-bade-a7a9cbae90a9 None admin] PATH_INFO: `/v3/projects/58e81a1daacf4bb08ea62cf133cf1073` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:58.369999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94d500-2153-4343-bade-a7a9cbae90a9 None admin] RBAC: Authorizing `identity:get_project(project_id=58e81a1daacf4bb08ea62cf133cf1073)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:58.371493 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94d500-2153-4343-bade-a7a9cbae90a9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:58.378043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 206/415] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:58 2022] GET /identity/v3/projects/58e81a1daacf4bb08ea62cf133cf1073 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:58.379201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94d500-2153-4343-bade-a7a9cbae90a9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:58.404956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2342102f-4c7f-49ba-8a52-e008deebefd1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:58.406097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2342102f-4c7f-49ba-8a52-e008deebefd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:58.440141 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:58.544120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 210/416] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:04:58.560713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94d500-2153-4343-bade-a7a9cbae90a9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:58.566489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342102f-4c7f-49ba-8a52-e008deebefd1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:58.653116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b74d5-2617-46a2-8ca2-efb14fac19f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:58.653815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b74d5-2617-46a2-8ca2-efb14fac19f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:58.654098 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b74d5-2617-46a2-8ca2-efb14fac19f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:58.654264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:58.654368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b74d5-2617-46a2-8ca2-efb14fac19f1 None admin] PATH_INFO: `/v3/projects/58e81a1daacf4bb08ea62cf133cf1073/groups/0d9aa4e1620a48fe9fd7e8dca2865e04/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:58.660670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b74d5-2617-46a2-8ca2-efb14fac19f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=58e81a1daacf4bb08ea62cf133cf1073, group_id=0d9aa4e1620a48fe9fd7e8dca2865e04, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:58.662651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b74d5-2617-46a2-8ca2-efb14fac19f1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:58.669832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342102f-4c7f-49ba-8a52-e008deebefd1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:58.705439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ee6bc-1c19-4454-92e9-8f70b7595ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:58.706081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ee6bc-1c19-4454-92e9-8f70b7595ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:58.706363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ee6bc-1c19-4454-92e9-8f70b7595ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:58.706642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ee6bc-1c19-4454-92e9-8f70b7595ce7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:58.717390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ee6bc-1c19-4454-92e9-8f70b7595ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-4f8ee6bc-1c19-4454-92e9-8f70b7595ce7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:58.722504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:58.723389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:04:58.724305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 211/417] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:04:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:04:58.730460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ee6bc-1c19-4454-92e9-8f70b7595ce7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:58.797013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 207/418] 10.222.0.25 () {58 vars in 1442 bytes} [Sat Aug 6 01:04:58 2022] PUT /identity/v3/projects/58e81a1daacf4bb08ea62cf133cf1073/groups/0d9aa4e1620a48fe9fd7e8dca2865e04/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:04:58.798415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c02f94-b07d-4083-a90d-94dcaabc6dc1 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:58.804004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c02f94-b07d-4083-a90d-94dcaabc6dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:58.804415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c02f94-b07d-4083-a90d-94dcaabc6dc1 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:58.806216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 208/419] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:58.812120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34911fc-dc24-4de6-a12d-935121ddbdf7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:58.812507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34911fc-dc24-4de6-a12d-935121ddbdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:58.812886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34911fc-dc24-4de6-a12d-935121ddbdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:58.847904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d34911fc-dc24-4de6-a12d-935121ddbdf7 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:58.849499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d34911fc-dc24-4de6-a12d-935121ddbdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:58.851402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ee6bc-1c19-4454-92e9-8f70b7595ce7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:58.897056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc856c-fb12-43ca-bb65-f1649630e96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:58.897697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc856c-fb12-43ca-bb65-f1649630e96a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:58.897983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc856c-fb12-43ca-bb65-f1649630e96a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:58.898262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc856c-fb12-43ca-bb65-f1649630e96a None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:58.899001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc856c-fb12-43ca-bb65-f1649630e96a None admin] RBAC: Adding query filter params (name=member) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:58.900052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc856c-fb12-43ca-bb65-f1649630e96a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:58.901663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc856c-fb12-43ca-bb65-f1649630e96a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:58.913261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 212/420] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:04:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:04:58.920910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc856c-fb12-43ca-bb65-f1649630e96a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:58.948200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:58.981833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 209/421] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:04:59.011478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34911fc-dc24-4de6-a12d-935121ddbdf7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:59.042207 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:59.110739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc856c-fb12-43ca-bb65-f1649630e96a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:59.139695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34911fc-dc24-4de6-a12d-935121ddbdf7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:59.152866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2b970-9fbd-440d-9540-a4537a6c67eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:59.153551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2b970-9fbd-440d-9540-a4537a6c67eb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:59.153836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2b970-9fbd-440d-9540-a4537a6c67eb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:59.154250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2b970-9fbd-440d-9540-a4537a6c67eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:59.182723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bd41f-4982-4b5c-8674-e4f8805a08e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:59.183581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bd41f-4982-4b5c-8674-e4f8805a08e0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:59.183892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bd41f-4982-4b5c-8674-e4f8805a08e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:59.184426 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bd41f-4982-4b5c-8674-e4f8805a08e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:59.199160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bd41f-4982-4b5c-8674-e4f8805a08e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d47bd41f-4982-4b5c-8674-e4f8805a08e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:04:59.206214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:04:59.208295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:04:59.210249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:04:59.215800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 210/422] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:04:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:04:59.215800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bd41f-4982-4b5c-8674-e4f8805a08e0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:59.236596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2b970-9fbd-440d-9540-a4537a6c67eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:59.238172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2b970-9fbd-440d-9540-a4537a6c67eb None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:59.250264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:59.333257 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:59.351906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bd41f-4982-4b5c-8674-e4f8805a08e0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:59.394185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055bccf-5981-49b0-90c7-f75e2dabc8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:59.395004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055bccf-5981-49b0-90c7-f75e2dabc8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:59.395294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055bccf-5981-49b0-90c7-f75e2dabc8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:59.395574 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055bccf-5981-49b0-90c7-f75e2dabc8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:59.396515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055bccf-5981-49b0-90c7-f75e2dabc8d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:59.396828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055bccf-5981-49b0-90c7-f75e2dabc8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:59.398519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055bccf-5981-49b0-90c7-f75e2dabc8d5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:59.408537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 211/423] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:04:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:59.414483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055bccf-5981-49b0-90c7-f75e2dabc8d5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:59.431988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 213/424] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:04:59.437640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2b970-9fbd-440d-9540-a4537a6c67eb None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:59.524731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055bccf-5981-49b0-90c7-f75e2dabc8d5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:59.558540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2b970-9fbd-440d-9540-a4537a6c67eb None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:59.562208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be882b-9208-4ff1-bff0-b9850dcf30de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:59.562852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be882b-9208-4ff1-bff0-b9850dcf30de None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:59.563130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be882b-9208-4ff1-bff0-b9850dcf30de None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:59.563412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be882b-9208-4ff1-bff0-b9850dcf30de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:59.568027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:59.591730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:59.594129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff38376-3d1d-4279-ae0b-61f8ab2997f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:59.594835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff38376-3d1d-4279-ae0b-61f8ab2997f0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:59.595175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff38376-3d1d-4279-ae0b-61f8ab2997f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:59.599952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff38376-3d1d-4279-ae0b-61f8ab2997f0 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:59.599952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff38376-3d1d-4279-ae0b-61f8ab2997f0 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:59.599952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff38376-3d1d-4279-ae0b-61f8ab2997f0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:59.601032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 214/425] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:59 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:59.606365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff38376-3d1d-4279-ae0b-61f8ab2997f0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:59.651417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be882b-9208-4ff1-bff0-b9850dcf30de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:59.653017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be882b-9208-4ff1-bff0-b9850dcf30de None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:59.709278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff38376-3d1d-4279-ae0b-61f8ab2997f0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:59.750239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd925cb-c7d1-4b2e-9f20-fbc9577b6d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:59.750876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd925cb-c7d1-4b2e-9f20-fbc9577b6d3a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:59.751167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd925cb-c7d1-4b2e-9f20-fbc9577b6d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:59.751426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd925cb-c7d1-4b2e-9f20-fbc9577b6d3a None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:59.754369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd925cb-c7d1-4b2e-9f20-fbc9577b6d3a None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:59.756011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd925cb-c7d1-4b2e-9f20-fbc9577b6d3a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:59.757930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 215/426] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:04:59 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:59.763633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd925cb-c7d1-4b2e-9f20-fbc9577b6d3a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:59.783220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 212/427] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:04:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 01:04:59.789491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be882b-9208-4ff1-bff0-b9850dcf30de None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:04:59.816607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:59.868879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd925cb-c7d1-4b2e-9f20-fbc9577b6d3a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:59.877366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:04:59.908605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81b7be-ca9b-4949-8d93-354607bda0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:59.908605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81b7be-ca9b-4949-8d93-354607bda0ea None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:59.908605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81b7be-ca9b-4949-8d93-354607bda0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:59.908605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81b7be-ca9b-4949-8d93-354607bda0ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:59.909553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81b7be-ca9b-4949-8d93-354607bda0ea None admin] RBAC: Adding query filter params (group.id=1e2af7c4177844c3a10e4d4665ecfff8, role.id=512a743b8ca748f8b793681748be7906, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:04:59.909553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81b7be-ca9b-4949-8d93-354607bda0ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:59.909553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81b7be-ca9b-4949-8d93-354607bda0ea None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:04:59.915742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 216/428] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Aug 6 01:04:59 2022] GET /identity/v3/role_assignments?group.id=1e2af7c4177844c3a10e4d4665ecfff8&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=512a743b8ca748f8b793681748be7906 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:04:59.921265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57676b72-9c9c-4362-8509-e3aa004f58c3 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:59.922747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57676b72-9c9c-4362-8509-e3aa004f58c3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:59.922747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57676b72-9c9c-4362-8509-e3aa004f58c3 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:59.923493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 217/429] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:04:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:04:59.929915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4f2f3-81a2-4680-88a0-63f854aed207 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:59.930288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4f2f3-81a2-4680-88a0-63f854aed207 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:59.930635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4f2f3-81a2-4680-88a0-63f854aed207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:59.945822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be882b-9208-4ff1-bff0-b9850dcf30de None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:04:59.965001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-dda4f2f3-81a2-4680-88a0-63f854aed207 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:04:59.966317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-dda4f2f3-81a2-4680-88a0-63f854aed207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:04:59.985323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1d3be-d5bb-421a-b34e-7b0f8d9627c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:04:59.985990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1d3be-d5bb-421a-b34e-7b0f8d9627c7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:04:59.987372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1d3be-d5bb-421a-b34e-7b0f8d9627c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:04:59.987657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1d3be-d5bb-421a-b34e-7b0f8d9627c7 None admin] PATH_INFO: `/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:04:59.991055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1d3be-d5bb-421a-b34e-7b0f8d9627c7 None admin] RBAC: Authorizing `identity:get_project(project_id=7fa1f9bede2243f0ba30c5f573dc695b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:04:59.992621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1d3be-d5bb-421a-b34e-7b0f8d9627c7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:00.003733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 213/430] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:04:59 2022] GET /identity/v3/projects/7fa1f9bede2243f0ba30c5f573dc695b => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:00.004726 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1d3be-d5bb-421a-b34e-7b0f8d9627c7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:00.148393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1d3be-d5bb-421a-b34e-7b0f8d9627c7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:00.152352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 218/431] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:05:00.175758 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:00.189878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4f2f3-81a2-4680-88a0-63f854aed207 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:00.204368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd1a74-a20f-4825-9bbc-bc17c0bd6066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:00.205330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1a74-a20f-4825-9bbc-bc17c0bd6066 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:00.205634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1a74-a20f-4825-9bbc-bc17c0bd6066 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:00.205921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1a74-a20f-4825-9bbc-bc17c0bd6066 None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:00.207309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd1a74-a20f-4825-9bbc-bc17c0bd6066 None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:00.209565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd1a74-a20f-4825-9bbc-bc17c0bd6066 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:00.211191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 214/432] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:05:00 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:00.217246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd1a74-a20f-4825-9bbc-bc17c0bd6066 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:00.261746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:00.337190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4f2f3-81a2-4680-88a0-63f854aed207 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:00.343928 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd1a74-a20f-4825-9bbc-bc17c0bd6066 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:00.386912 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f63a1-b212-46ce-9311-ad76ddc87955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:00.387941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f63a1-b212-46ce-9311-ad76ddc87955 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:00.388419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f63a1-b212-46ce-9311-ad76ddc87955 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:00.389069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f63a1-b212-46ce-9311-ad76ddc87955 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:00.389410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383984f8-42d1-41f7-891f-662a2acbb2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:00.390045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383984f8-42d1-41f7-891f-662a2acbb2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:00.390344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383984f8-42d1-41f7-891f-662a2acbb2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:00.390867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383984f8-42d1-41f7-891f-662a2acbb2b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:00.391716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383984f8-42d1-41f7-891f-662a2acbb2b4 None admin] RBAC: Adding query filter params (group.id=1e2af7c4177844c3a10e4d4665ecfff8, role.id=b5b241afd53c456e9a9fa3033eea74ad, scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:00.392069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383984f8-42d1-41f7-891f-662a2acbb2b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:00.394068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383984f8-42d1-41f7-891f-662a2acbb2b4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:00.405920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 215/433] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Aug 6 01:05:00 2022] GET /identity/v3/role_assignments?group.id=1e2af7c4177844c3a10e4d4665ecfff8&scope.project.id=7fa1f9bede2243f0ba30c5f573dc695b&role.id=b5b241afd53c456e9a9fa3033eea74ad => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:00.406334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f63a1-b212-46ce-9311-ad76ddc87955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8c4f63a1-b212-46ce-9311-ad76ddc87955 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:00.411565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:00.412568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 01:05:00.413389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:00.416268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 219/434] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:00.422383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383984f8-42d1-41f7-891f-662a2acbb2b4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:00.493005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:00.525930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce8fb7-877f-4445-839b-6d7029b7219e None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:00.526293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce8fb7-877f-4445-839b-6d7029b7219e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:00.526516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce8fb7-877f-4445-839b-6d7029b7219e None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:00.532573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 220/435] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:05:00.535705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7590e-84aa-4cca-9aee-c552b56d6826 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:00.535881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7590e-84aa-4cca-9aee-c552b56d6826 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:00.536081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7590e-84aa-4cca-9aee-c552b56d6826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:00.558469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-57e7590e-84aa-4cca-9aee-c552b56d6826 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:00.559682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-57e7590e-84aa-4cca-9aee-c552b56d6826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:00.560421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383984f8-42d1-41f7-891f-662a2acbb2b4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:00.605475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3573b-8318-4fad-9e5c-2f22b5cf9dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:00.606112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3573b-8318-4fad-9e5c-2f22b5cf9dcd None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:00.606397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3573b-8318-4fad-9e5c-2f22b5cf9dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:00.606675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3573b-8318-4fad-9e5c-2f22b5cf9dcd None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:00.607404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3573b-8318-4fad-9e5c-2f22b5cf9dcd None admin] RBAC: Adding query filter params (name=member) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:00.607713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3573b-8318-4fad-9e5c-2f22b5cf9dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:00.609145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3573b-8318-4fad-9e5c-2f22b5cf9dcd None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:00.620331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 216/436] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:05:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:00.626840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3573b-8318-4fad-9e5c-2f22b5cf9dcd None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:00.667717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 221/437] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:00.692699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7590e-84aa-4cca-9aee-c552b56d6826 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:00.728010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:00.752383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3573b-8318-4fad-9e5c-2f22b5cf9dcd None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:00.764227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:00.794545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5daf05-ffc1-43ce-a458-4176024b62fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:00.795185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5daf05-ffc1-43ce-a458-4176024b62fb None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:00.795475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5daf05-ffc1-43ce-a458-4176024b62fb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:00.795754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5daf05-ffc1-43ce-a458-4176024b62fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:00.829568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7590e-84aa-4cca-9aee-c552b56d6826 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:00.877356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ce70d-44c4-4402-a080-fc8112780978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:00.878008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ce70d-44c4-4402-a080-fc8112780978 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:00.878286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ce70d-44c4-4402-a080-fc8112780978 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:00.878556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ce70d-44c4-4402-a080-fc8112780978 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:00.891313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5daf05-ffc1-43ce-a458-4176024b62fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:00.893290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5daf05-ffc1-43ce-a458-4176024b62fb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:00.897707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ce70d-44c4-4402-a080-fc8112780978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-af2ce70d-44c4-4402-a080-fc8112780978 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:00.902329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:00.903215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:00.904044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 222/438] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:00.911915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ce70d-44c4-4402-a080-fc8112780978 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:01.011891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 217/439] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:01.016655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5daf05-ffc1-43ce-a458-4176024b62fb None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:01.026698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:01.027122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:01.040905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ce70d-44c4-4402-a080-fc8112780978 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:01.076701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622da8d-b36d-41b6-8bf4-c804d6373d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:01.077485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622da8d-b36d-41b6-8bf4-c804d6373d18 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.077770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622da8d-b36d-41b6-8bf4-c804d6373d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.078050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622da8d-b36d-41b6-8bf4-c804d6373d18 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.078769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622da8d-b36d-41b6-8bf4-c804d6373d18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:01.079076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622da8d-b36d-41b6-8bf4-c804d6373d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:01.080465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622da8d-b36d-41b6-8bf4-c804d6373d18 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:01.093552 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 223/440] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:01.094826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622da8d-b36d-41b6-8bf4-c804d6373d18 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:01.120867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5daf05-ffc1-43ce-a458-4176024b62fb None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:01.156237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e04f5-ced1-4b1b-93db-a9ff1dd16be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:01.157106 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e04f5-ced1-4b1b-93db-a9ff1dd16be6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.157592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e04f5-ced1-4b1b-93db-a9ff1dd16be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.157873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e04f5-ced1-4b1b-93db-a9ff1dd16be6 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.159032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e04f5-ced1-4b1b-93db-a9ff1dd16be6 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:01.160502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e04f5-ced1-4b1b-93db-a9ff1dd16be6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:01.162359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 218/441] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:05:01 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:01.167473 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e04f5-ced1-4b1b-93db-a9ff1dd16be6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:01.202803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622da8d-b36d-41b6-8bf4-c804d6373d18 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:01.237490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b82a9-aee4-4080-b5aa-b7da2638f05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:01.238112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b82a9-aee4-4080-b5aa-b7da2638f05b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.238397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b82a9-aee4-4080-b5aa-b7da2638f05b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.238677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b82a9-aee4-4080-b5aa-b7da2638f05b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.258194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:01.259145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:01.281299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e04f5-ced1-4b1b-93db-a9ff1dd16be6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:01.307971 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b82a9-aee4-4080-b5aa-b7da2638f05b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:01.309421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b82a9-aee4-4080-b5aa-b7da2638f05b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:01.316946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f66561-c44d-4df9-860d-287faa962321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:01.317588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f66561-c44d-4df9-860d-287faa962321 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.317867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f66561-c44d-4df9-860d-287faa962321 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.318658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f66561-c44d-4df9-860d-287faa962321 None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.319810 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f66561-c44d-4df9-860d-287faa962321 None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:01.321251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f66561-c44d-4df9-860d-287faa962321 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:01.327806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 219/442] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:05:01 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:01.329177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f66561-c44d-4df9-860d-287faa962321 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:01.417595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 224/443] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 01:05:01.422986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b82a9-aee4-4080-b5aa-b7da2638f05b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:01.449009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f66561-c44d-4df9-860d-287faa962321 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:01.471804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:01.482211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb56ec-ba8c-4b25-bec1-5ac5d6e39ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:01.483931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb56ec-ba8c-4b25-bec1-5ac5d6e39ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.483931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb56ec-ba8c-4b25-bec1-5ac5d6e39ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.483931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb56ec-ba8c-4b25-bec1-5ac5d6e39ee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.484202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb56ec-ba8c-4b25-bec1-5ac5d6e39ee8 None admin] RBAC: Adding query filter params (group.id=1e2af7c4177844c3a10e4d4665ecfff8, role.id=512a743b8ca748f8b793681748be7906, scope.project.id=3059810e56d143ab91728e2220010d53) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:01.484456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb56ec-ba8c-4b25-bec1-5ac5d6e39ee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:01.485894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb56ec-ba8c-4b25-bec1-5ac5d6e39ee8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:01.492012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 220/444] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Aug 6 01:05:01 2022] GET /identity/v3/role_assignments?group.id=1e2af7c4177844c3a10e4d4665ecfff8&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=512a743b8ca748f8b793681748be7906 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:01.493151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016bc4a-14cf-42e3-bdaa-26c42d0e0c44 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.493439 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016bc4a-14cf-42e3-bdaa-26c42d0e0c44 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.493718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016bc4a-14cf-42e3-bdaa-26c42d0e0c44 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.495829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 221/445] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:01.500701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95cc27-2fef-4624-be82-d5bfb82b4366 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.501324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95cc27-2fef-4624-be82-d5bfb82b4366 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.501608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d95cc27-2fef-4624-be82-d5bfb82b4366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.525013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5d95cc27-2fef-4624-be82-d5bfb82b4366 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:01.526659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5d95cc27-2fef-4624-be82-d5bfb82b4366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:01.527209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b82a9-aee4-4080-b5aa-b7da2638f05b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:01.529806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:01.566491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130d1f9-8da7-498a-8954-ecd128d7fb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:01.567056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130d1f9-8da7-498a-8954-ecd128d7fb80 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.567331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130d1f9-8da7-498a-8954-ecd128d7fb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.567631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130d1f9-8da7-498a-8954-ecd128d7fb80 None admin] PATH_INFO: `/v3/projects/3059810e56d143ab91728e2220010d53` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.569385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130d1f9-8da7-498a-8954-ecd128d7fb80 None admin] RBAC: Authorizing `identity:get_project(project_id=3059810e56d143ab91728e2220010d53)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:01.572026 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130d1f9-8da7-498a-8954-ecd128d7fb80 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:01.573825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 225/446] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:05:01 2022] GET /identity/v3/projects/3059810e56d143ab91728e2220010d53 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:01.582614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130d1f9-8da7-498a-8954-ecd128d7fb80 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:01.613557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 222/447] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:01.638483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95cc27-2fef-4624-be82-d5bfb82b4366 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:01.712116 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130d1f9-8da7-498a-8954-ecd128d7fb80 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:01.733614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:01.748900 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d95cc27-2fef-4624-be82-d5bfb82b4366 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:01.753443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd79ab-ecc7-4788-8150-ea8bea98629f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:01.756440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd79ab-ecc7-4788-8150-ea8bea98629f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.756440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd79ab-ecc7-4788-8150-ea8bea98629f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.756440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd79ab-ecc7-4788-8150-ea8bea98629f None admin] PATH_INFO: `/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.756440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd79ab-ecc7-4788-8150-ea8bea98629f None admin] RBAC: Authorizing `identity:get_group(group_id=1e2af7c4177844c3a10e4d4665ecfff8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:01.758859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd79ab-ecc7-4788-8150-ea8bea98629f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:01.760164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 226/448] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:05:01 2022] GET /identity/v3/groups/1e2af7c4177844c3a10e4d4665ecfff8 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:01.766153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd79ab-ecc7-4788-8150-ea8bea98629f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:01.797904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd8ec8-02a7-407b-87f9-c78e59932ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:01.798559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd8ec8-02a7-407b-87f9-c78e59932ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.798836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd8ec8-02a7-407b-87f9-c78e59932ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.799214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd8ec8-02a7-407b-87f9-c78e59932ee8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.808552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:01.812197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd8ec8-02a7-407b-87f9-c78e59932ee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4ccd8ec8-02a7-407b-87f9-c78e59932ee8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:01.823850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:01.824969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:01.826077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 223/449] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:01.828204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd8ec8-02a7-407b-87f9-c78e59932ee8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:01.879756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd79ab-ecc7-4788-8150-ea8bea98629f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:01.916993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2124b-4a21-41f7-a951-bd3ed642ba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:01.918559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2124b-4a21-41f7-a951-bd3ed642ba87 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.918848 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2124b-4a21-41f7-a951-bd3ed642ba87 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.919130 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2124b-4a21-41f7-a951-bd3ed642ba87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.919973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2124b-4a21-41f7-a951-bd3ed642ba87 None admin] RBAC: Adding query filter params (group.id=1e2af7c4177844c3a10e4d4665ecfff8, role.id=b5b241afd53c456e9a9fa3033eea74ad, scope.project.id=3059810e56d143ab91728e2220010d53) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:01.920292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2124b-4a21-41f7-a951-bd3ed642ba87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:01.921751 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2124b-4a21-41f7-a951-bd3ed642ba87 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:01.929114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 227/450] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Aug 6 01:05:01 2022] GET /identity/v3/role_assignments?group.id=1e2af7c4177844c3a10e4d4665ecfff8&scope.project.id=3059810e56d143ab91728e2220010d53&role.id=b5b241afd53c456e9a9fa3033eea74ad => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:01.934917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd8ec8-02a7-407b-87f9-c78e59932ee8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:01.970866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb1d73-68df-4b1d-9d8e-3d07656e1ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:01.971504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1d73-68df-4b1d-9d8e-3d07656e1ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:01.971810 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1d73-68df-4b1d-9d8e-3d07656e1ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:01.972089 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1d73-68df-4b1d-9d8e-3d07656e1ed8 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:01.972826 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1d73-68df-4b1d-9d8e-3d07656e1ed8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:01.973138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1d73-68df-4b1d-9d8e-3d07656e1ed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:01.975109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1d73-68df-4b1d-9d8e-3d07656e1ed8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:01.984605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 224/451] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:01.990873 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2124b-4a21-41f7-a951-bd3ed642ba87 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:02.042716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:02.094733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2124b-4a21-41f7-a951-bd3ed642ba87 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:02.128313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bae73-d7a9-480e-8152-966856154500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:02.128930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bae73-d7a9-480e-8152-966856154500 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:02.129214 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bae73-d7a9-480e-8152-966856154500 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:02.129491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bae73-d7a9-480e-8152-966856154500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:02.199890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bae73-d7a9-480e-8152-966856154500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:02.201613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bae73-d7a9-480e-8152-966856154500 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:02.249284 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:02.299205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 228/452] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:02.304941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb1d73-68df-4b1d-9d8e-3d07656e1ed8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:02.361913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:02.400566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb1d73-68df-4b1d-9d8e-3d07656e1ed8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:02.433147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e520a-17d8-4607-acbe-24791dd72b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:02.434227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e520a-17d8-4607-acbe-24791dd72b66 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:02.434702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e520a-17d8-4607-acbe-24791dd72b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:02.434991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e520a-17d8-4607-acbe-24791dd72b66 None admin] PATH_INFO: `/v3/projects/58e81a1daacf4bb08ea62cf133cf1073` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:02.436212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e520a-17d8-4607-acbe-24791dd72b66 None admin] RBAC: Authorizing `identity:get_project(project_id=58e81a1daacf4bb08ea62cf133cf1073)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:02.437663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e520a-17d8-4607-acbe-24791dd72b66 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:02.439339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 225/453] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:05:02 2022] GET /identity/v3/projects/58e81a1daacf4bb08ea62cf133cf1073 => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:02.445081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bae73-d7a9-480e-8152-966856154500 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:02.540705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bae73-d7a9-480e-8152-966856154500 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:02.572559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef7daa-5cae-47d2-855f-652392ddf09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:02.573178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef7daa-5cae-47d2-855f-652392ddf09c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:02.573455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef7daa-5cae-47d2-855f-652392ddf09c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:02.573724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef7daa-5cae-47d2-855f-652392ddf09c None admin] PATH_INFO: `/v3/groups/0d9aa4e1620a48fe9fd7e8dca2865e04` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:02.574831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef7daa-5cae-47d2-855f-652392ddf09c None admin] RBAC: Authorizing `identity:get_group(group_id=0d9aa4e1620a48fe9fd7e8dca2865e04)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:02.576291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef7daa-5cae-47d2-855f-652392ddf09c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:02.578107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 229/454] 10.222.0.25 () {56 vars in 1180 bytes} [Sat Aug 6 01:05:02 2022] GET /identity/v3/groups/0d9aa4e1620a48fe9fd7e8dca2865e04 => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:02.584518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e520a-17d8-4607-acbe-24791dd72b66 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:02.678258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e520a-17d8-4607-acbe-24791dd72b66 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:02.703123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:02.710799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f4653-212a-44a3-879f-0944816aa561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:02.711400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f4653-212a-44a3-879f-0944816aa561 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:02.711679 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f4653-212a-44a3-879f-0944816aa561 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:02.711979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f4653-212a-44a3-879f-0944816aa561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:02.712809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f4653-212a-44a3-879f-0944816aa561 None admin] RBAC: Adding query filter params (group.id=0d9aa4e1620a48fe9fd7e8dca2865e04, role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=58e81a1daacf4bb08ea62cf133cf1073) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:02.713113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f4653-212a-44a3-879f-0944816aa561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:02.714534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f4653-212a-44a3-879f-0944816aa561 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:02.719993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 226/455] 10.222.0.25 () {56 vars in 1376 bytes} [Sat Aug 6 01:05:02 2022] GET /identity/v3/role_assignments?group.id=0d9aa4e1620a48fe9fd7e8dca2865e04&scope.project.id=58e81a1daacf4bb08ea62cf133cf1073&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:05.098138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94056-eab0-4ff7-b0b3-8d496eac91a9 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:05.098661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94056-eab0-4ff7-b0b3-8d496eac91a9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:05.098764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94056-eab0-4ff7-b0b3-8d496eac91a9 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:05.100342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 230/456] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:05.105677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d523-1212-4b1a-b3ce-5a8137569299 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:05.106488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d523-1212-4b1a-b3ce-5a8137569299 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:05.106979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d523-1212-4b1a-b3ce-5a8137569299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:05.125045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-efe1d523-1212-4b1a-b3ce-5a8137569299 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:05.126530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-efe1d523-1212-4b1a-b3ce-5a8137569299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:05.210872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 227/457] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:05.235159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed94056-eab0-4ff7-b0b3-8d496eac91a9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:05.260797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:05.368542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed94056-eab0-4ff7-b0b3-8d496eac91a9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:05.417821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edefa66-32e2-4328-bf3a-6e5c6e74ac95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:05.418837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edefa66-32e2-4328-bf3a-6e5c6e74ac95 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:05.419337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edefa66-32e2-4328-bf3a-6e5c6e74ac95 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:05.419858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edefa66-32e2-4328-bf3a-6e5c6e74ac95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:05.433335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edefa66-32e2-4328-bf3a-6e5c6e74ac95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:05.435206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edefa66-32e2-4328-bf3a-6e5c6e74ac95 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8edefa66-32e2-4328-bf3a-6e5c6e74ac95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:05.455469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:05.456452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:05.457326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:05.457326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:05.457326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:05.457326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:05.459502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 231/458] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:05.465665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1d523-1212-4b1a-b3ce-5a8137569299 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:05.608925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1d523-1212-4b1a-b3ce-5a8137569299 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:05.649866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:05.665507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167a357-02b2-45d7-9678-a0fe4f508c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:05.666145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167a357-02b2-45d7-9678-a0fe4f508c66 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:05.669303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167a357-02b2-45d7-9678-a0fe4f508c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:05.669303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167a357-02b2-45d7-9678-a0fe4f508c66 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:05.669303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167a357-02b2-45d7-9678-a0fe4f508c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:05.669303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167a357-02b2-45d7-9678-a0fe4f508c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:05.669303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167a357-02b2-45d7-9678-a0fe4f508c66 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:05.689911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 228/459] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:05.695380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edefa66-32e2-4328-bf3a-6e5c6e74ac95 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:05.740769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbe941-a2b5-427b-adfe-fd3245451779 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:05.741084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbe941-a2b5-427b-adfe-fd3245451779 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:05.741894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbe941-a2b5-427b-adfe-fd3245451779 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:05.743309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 229/460] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:05:05.744712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50837ce-50a8-41c2-9c24-52e990910f81 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:05.744997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50837ce-50a8-41c2-9c24-52e990910f81 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:05.745278 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50837ce-50a8-41c2-9c24-52e990910f81 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:05.746696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 230/461] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:05:05.748814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d149d6-3b2d-41ed-86d3-1a7bb49d62f1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:05.749100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d149d6-3b2d-41ed-86d3-1a7bb49d62f1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:05.749377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d149d6-3b2d-41ed-86d3-1a7bb49d62f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:05.779320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-78d149d6-3b2d-41ed-86d3-1a7bb49d62f1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:05.780837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-78d149d6-3b2d-41ed-86d3-1a7bb49d62f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:05.781638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:05.819601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edefa66-32e2-4328-bf3a-6e5c6e74ac95 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:05.869729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6095078-9586-4785-8743-ef839226f924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:05.875307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6095078-9586-4785-8743-ef839226f924 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:05.875452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 231/462] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 06 01:05:05.875452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686883b-7aca-4e72-b9ab-c2d0208cf9de None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:05.875452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686883b-7aca-4e72-b9ab-c2d0208cf9de None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:05.875452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686883b-7aca-4e72-b9ab-c2d0208cf9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:05.880582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6095078-9586-4785-8743-ef839226f924 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:05.881106 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6095078-9586-4785-8743-ef839226f924 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:05.882070 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6095078-9586-4785-8743-ef839226f924 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:05.883811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6095078-9586-4785-8743-ef839226f924 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:05.910599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4686883b-7aca-4e72-b9ab-c2d0208cf9de None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:05.911820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4686883b-7aca-4e72-b9ab-c2d0208cf9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:05.985281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 232/463] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:05.986122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686883b-7aca-4e72-b9ab-c2d0208cf9de None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:06.010521 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 232/464] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:05:05 2022] POST /identity/v3/users => generated 246 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:06.013355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3fe5e-f666-4804-9f60-ec61f60bcd79 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.013663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3fe5e-f666-4804-9f60-ec61f60bcd79 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.014415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3fe5e-f666-4804-9f60-ec61f60bcd79 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.017045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 233/465] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:06.018202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3fe5e-f666-4804-9f60-ec61f60bcd79 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:06.037563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:06.114316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686883b-7aca-4e72-b9ab-c2d0208cf9de None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:06.156794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3fe5e-f666-4804-9f60-ec61f60bcd79 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:06.162515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a357925-770d-4038-aac0-268c524c4232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:06.163769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a357925-770d-4038-aac0-268c524c4232 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.164143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a357925-770d-4038-aac0-268c524c4232 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.164486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a357925-770d-4038-aac0-268c524c4232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.180192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a357925-770d-4038-aac0-268c524c4232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:06.181760 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a357925-770d-4038-aac0-268c524c4232 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:06.189793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:06.199536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e14bf-6aa9-4950-a2ca-91b886ce90ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:06.200220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e14bf-6aa9-4950-a2ca-91b886ce90ad None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.200501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e14bf-6aa9-4950-a2ca-91b886ce90ad None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.200776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e14bf-6aa9-4950-a2ca-91b886ce90ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-2a357925-770d-4038-aac0-268c524c4232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:06.202177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:06.202991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:06.208599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:06.208599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:06.208599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:06.208599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:06.208599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 233/466] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:06.208599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a684f5-d4c9-4233-a37d-5fd7b28cb22a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.208599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a684f5-d4c9-4233-a37d-5fd7b28cb22a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.208599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a684f5-d4c9-4233-a37d-5fd7b28cb22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.213822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e14bf-6aa9-4950-a2ca-91b886ce90ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:06.216109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e14bf-6aa9-4950-a2ca-91b886ce90ad None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:06.228651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-94a684f5-d4c9-4233-a37d-5fd7b28cb22a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:06.229308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-94a684f5-d4c9-4233-a37d-5fd7b28cb22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-a38e14bf-6aa9-4950-a2ca-91b886ce90ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:06.238208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:06.240411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:06.241431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:06.241431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:06.241431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:06.241431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:06.241431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 234/467] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:06.241697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e14bf-6aa9-4950-a2ca-91b886ce90ad None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:06.316708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 234/468] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:06.318426 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a684f5-d4c9-4233-a37d-5fd7b28cb22a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:06.347986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:06.387170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e14bf-6aa9-4950-a2ca-91b886ce90ad None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:06.423392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9891-0028-4290-9a38-a56567fab411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:06.424073 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9891-0028-4290-9a38-a56567fab411 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.424430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9891-0028-4290-9a38-a56567fab411 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.424782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9891-0028-4290-9a38-a56567fab411 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.425553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f9891-0028-4290-9a38-a56567fab411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:06.425932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f9891-0028-4290-9a38-a56567fab411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:06.427377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f9891-0028-4290-9a38-a56567fab411 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:06.439002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a684f5-d4c9-4233-a37d-5fd7b28cb22a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:06.445171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 235/469] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:06.445171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9891-0028-4290-9a38-a56567fab411 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:06.475393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b33ab4-b112-49f6-bcc1-f75608ea9e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:06.476086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b33ab4-b112-49f6-bcc1-f75608ea9e36 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.476365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b33ab4-b112-49f6-bcc1-f75608ea9e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.476639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b33ab4-b112-49f6-bcc1-f75608ea9e36 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.477324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b33ab4-b112-49f6-bcc1-f75608ea9e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:06.477626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b33ab4-b112-49f6-bcc1-f75608ea9e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:06.478992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b33ab4-b112-49f6-bcc1-f75608ea9e36 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:06.486345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:06.490700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 235/470] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:06.492036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0054c-719c-49f9-af21-5671f2e299ea None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.492357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0054c-719c-49f9-af21-5671f2e299ea None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.492631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0054c-719c-49f9-af21-5671f2e299ea None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.493988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 236/471] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:06.494483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0054c-719c-49f9-af21-5671f2e299ea None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:06.569908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9891-0028-4290-9a38-a56567fab411 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:06.589315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:06.606562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0054c-719c-49f9-af21-5671f2e299ea None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:06.610939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f8de5-aa00-49d5-8bff-5669936b2341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:06.611676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f8de5-aa00-49d5-8bff-5669936b2341 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.611980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f8de5-aa00-49d5-8bff-5669936b2341 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.612351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f8de5-aa00-49d5-8bff-5669936b2341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.623527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f8de5-aa00-49d5-8bff-5669936b2341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:06.625001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f8de5-aa00-49d5-8bff-5669936b2341 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-cb1f8de5-aa00-49d5-8bff-5669936b2341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:06.640467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:06.641372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:06.642227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:06.642227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:06.642227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:06.642227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:06.644139 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fb407-2ba5-4969-a799-a2ebe78db25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:06.644773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fb407-2ba5-4969-a799-a2ebe78db25f None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.645058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fb407-2ba5-4969-a799-a2ebe78db25f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.645336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fb407-2ba5-4969-a799-a2ebe78db25f None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.646041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fb407-2ba5-4969-a799-a2ebe78db25f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:06.646515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 236/472] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:06.647156 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f8de5-aa00-49d5-8bff-5669936b2341 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:06.648409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fb407-2ba5-4969-a799-a2ebe78db25f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:06.729113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 237/473] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:05:06 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:06.729113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ed8dd-742c-46db-b351-d2e8c67380da None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.729113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ed8dd-742c-46db-b351-d2e8c67380da None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.729113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ed8dd-742c-46db-b351-d2e8c67380da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.744027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a42ed8dd-742c-46db-b351-d2e8c67380da None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:06.745398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a42ed8dd-742c-46db-b351-d2e8c67380da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:06.751191 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:06.763134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f8de5-aa00-49d5-8bff-5669936b2341 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:06.801347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfe6a5-ad00-4a2a-806d-3a4281a1cfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:06.802070 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfe6a5-ad00-4a2a-806d-3a4281a1cfc1 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:06.802509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfe6a5-ad00-4a2a-806d-3a4281a1cfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:06.802872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfe6a5-ad00-4a2a-806d-3a4281a1cfc1 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:06.803624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfe6a5-ad00-4a2a-806d-3a4281a1cfc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:06.805811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfe6a5-ad00-4a2a-806d-3a4281a1cfc1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:06.835252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 238/474] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:06.839836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ed8dd-742c-46db-b351-d2e8c67380da None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:06.910439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 237/475] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:05:06 2022] POST /identity/v3/users => generated 248 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:06.911500 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfe6a5-ad00-4a2a-806d-3a4281a1cfc1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:06.948284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:06.974300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ed8dd-742c-46db-b351-d2e8c67380da None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:07.013360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f658b-6280-4c3d-baa7-f904e8380535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:07.013913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f658b-6280-4c3d-baa7-f904e8380535 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:07.014192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f658b-6280-4c3d-baa7-f904e8380535 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:07.014467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f658b-6280-4c3d-baa7-f904e8380535 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:07.015315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f658b-6280-4c3d-baa7-f904e8380535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:07.015702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f658b-6280-4c3d-baa7-f904e8380535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:07.017164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f658b-6280-4c3d-baa7-f904e8380535 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:07.030182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 239/476] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:07.035852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f658b-6280-4c3d-baa7-f904e8380535 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:07.058177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfe6a5-ad00-4a2a-806d-3a4281a1cfc1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:07.116114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558092f-afa9-4e40-bcc3-482c3a1b010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:07.117330 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558092f-afa9-4e40-bcc3-482c3a1b010a None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:07.117620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558092f-afa9-4e40-bcc3-482c3a1b010a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:07.117903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558092f-afa9-4e40-bcc3-482c3a1b010a None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:07.118746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558092f-afa9-4e40-bcc3-482c3a1b010a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:07.120306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558092f-afa9-4e40-bcc3-482c3a1b010a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:07.162998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.sql.core [None req-9558092f-afa9-4e40-bcc3-482c3a1b010a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 01:05:07.162998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 06 01:05:07.162998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [parameters: {'id': '33e199d66e774ad9b8f0632d107fec6a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 06 01:05:07.162998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80525) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 01:05:07.166131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f658b-6280-4c3d-baa7-f904e8380535 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9558092f-afa9-4e40-bcc3-482c3a1b010a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application result.read() Aug 06 01:05:07.183098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return ref.to_dict() Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:05:07.184962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application yield resource Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise self.value Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application session.commit() Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 01:05:07.185915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self.session.flush() Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application compat.raise_( Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 01:05:07.186791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:05:07.187695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application result.read() Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:05:07.188563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [parameters: {'id': '33e199d66e774ad9b8f0632d107fec6a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:07.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 06 01:05:07.193126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:07.196983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 238/477] 10.222.0.25 () {60 vars in 1131 bytes} [Sat Aug 6 01:05:06 2022] POST /identity/v3/roles => generated 144 bytes in 286 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:07.197562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:07.202748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558092f-afa9-4e40-bcc3-482c3a1b010a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:07.206446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5f9fa-3c5c-4084-8923-74bb8b812fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:07.207352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5f9fa-3c5c-4084-8923-74bb8b812fa1 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:07.209476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5f9fa-3c5c-4084-8923-74bb8b812fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:07.209476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5f9fa-3c5c-4084-8923-74bb8b812fa1 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:07.209802 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5f9fa-3c5c-4084-8923-74bb8b812fa1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:07.211543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5f9fa-3c5c-4084-8923-74bb8b812fa1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:07.219728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:07.289481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 240/478] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:05:07 2022] POST /identity/v3/users => generated 249 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:05:07.335381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558092f-afa9-4e40-bcc3-482c3a1b010a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:07.375597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c72ed0-5d46-4845-8024-350d84fdb321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:07.376684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c72ed0-5d46-4845-8024-350d84fdb321 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:07.377172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c72ed0-5d46-4845-8024-350d84fdb321 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:07.377651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c72ed0-5d46-4845-8024-350d84fdb321 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:07.390208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c72ed0-5d46-4845-8024-350d84fdb321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-80c72ed0-5d46-4845-8024-350d84fdb321 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:07.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:07.397308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 01:05:07.398257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:07.401190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 239/479] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:07.408940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5f9fa-3c5c-4084-8923-74bb8b812fa1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:07.534739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5f9fa-3c5c-4084-8923-74bb8b812fa1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:07.572220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec99dc2-672d-4bf3-b3c3-cdcf1dba9e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:07.572776 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec99dc2-672d-4bf3-b3c3-cdcf1dba9e00 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:07.573055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec99dc2-672d-4bf3-b3c3-cdcf1dba9e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:07.573328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec99dc2-672d-4bf3-b3c3-cdcf1dba9e00 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:07.574044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec99dc2-672d-4bf3-b3c3-cdcf1dba9e00 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:07.574347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec99dc2-672d-4bf3-b3c3-cdcf1dba9e00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:07.576246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec99dc2-672d-4bf3-b3c3-cdcf1dba9e00 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:07.585802 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 241/480] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:08.318392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbe574-098e-4d36-96db-24e61d3f2f75 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:08.320039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbe574-098e-4d36-96db-24e61d3f2f75 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:08.320535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbe574-098e-4d36-96db-24e61d3f2f75 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:08.322340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 240/481] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:08.328152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc39eca-8a45-4775-b2a2-e276b9d094cf None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:08.328541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc39eca-8a45-4775-b2a2-e276b9d094cf None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:08.328727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc39eca-8a45-4775-b2a2-e276b9d094cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:08.356774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4dc39eca-8a45-4775-b2a2-e276b9d094cf None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:08.357978 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4dc39eca-8a45-4775-b2a2-e276b9d094cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:08.417282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:08.440760 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 242/482] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:08.476204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbe574-098e-4d36-96db-24e61d3f2f75 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:08.571549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:08.620565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbe574-098e-4d36-96db-24e61d3f2f75 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:08.665374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3b8c0-c999-4651-8b46-893973b88ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:08.666021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3b8c0-c999-4651-8b46-893973b88ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:08.666302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3b8c0-c999-4651-8b46-893973b88ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:08.666575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3b8c0-c999-4651-8b46-893973b88ac9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:08.682563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3b8c0-c999-4651-8b46-893973b88ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-eba3b8c0-c999-4651-8b46-893973b88ac9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:08.690154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:08.691127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:08.692319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 241/483] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:08.697269 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc39eca-8a45-4775-b2a2-e276b9d094cf None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:08.828622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc39eca-8a45-4775-b2a2-e276b9d094cf None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:08.874463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6283aab-a9e2-4f26-b5a0-8ac1f4c006a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:08.878358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6283aab-a9e2-4f26-b5a0-8ac1f4c006a2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:08.878358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6283aab-a9e2-4f26-b5a0-8ac1f4c006a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:08.878358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6283aab-a9e2-4f26-b5a0-8ac1f4c006a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:08.878358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6283aab-a9e2-4f26-b5a0-8ac1f4c006a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:08.878358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6283aab-a9e2-4f26-b5a0-8ac1f4c006a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:08.878358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6283aab-a9e2-4f26-b5a0-8ac1f4c006a2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:08.891946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 243/484] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:08.897619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3b8c0-c999-4651-8b46-893973b88ac9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:09.056564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9b147-f69f-4b96-a9e1-2680cbe875b0 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:09.056928 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9b147-f69f-4b96-a9e1-2680cbe875b0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:09.057124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9b147-f69f-4b96-a9e1-2680cbe875b0 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:09.057504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3b8c0-c999-4651-8b46-893973b88ac9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:09.058689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 244/485] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:05:09.063730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb0bb-48f1-440b-9838-ab3366ab598f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:09.064045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb0bb-48f1-440b-9838-ab3366ab598f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:09.064324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb0bb-48f1-440b-9838-ab3366ab598f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:09.094377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-24dfb0bb-48f1-440b-9838-ab3366ab598f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:09.095548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-24dfb0bb-48f1-440b-9838-ab3366ab598f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:09.114876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:09.124282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b51dd-7535-4eff-9701-1d0af716cbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:09.124941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b51dd-7535-4eff-9701-1d0af716cbef None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:09.125220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b51dd-7535-4eff-9701-1d0af716cbef None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:09.125496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b51dd-7535-4eff-9701-1d0af716cbef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:09.144577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b51dd-7535-4eff-9701-1d0af716cbef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:09.146032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b51dd-7535-4eff-9701-1d0af716cbef None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:09.168135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9b8b51dd-7535-4eff-9701-1d0af716cbef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:09.176750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:09.177610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:09.178468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:09.178468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:09.178468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:09.178468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:09.180084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 242/486] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:09.184646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b51dd-7535-4eff-9701-1d0af716cbef None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:09.227108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 245/487] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:05:09.280053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfb0bb-48f1-440b-9838-ab3366ab598f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:09.370375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b51dd-7535-4eff-9701-1d0af716cbef None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:09.416805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfb0bb-48f1-440b-9838-ab3366ab598f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:09.444134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f8e3-debd-403e-843b-0e9ba4d1fc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:09.445297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f8e3-debd-403e-843b-0e9ba4d1fc4f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:09.445578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f8e3-debd-403e-843b-0e9ba4d1fc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:09.445855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924f8e3-debd-403e-843b-0e9ba4d1fc4f None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:09.446551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924f8e3-debd-403e-843b-0e9ba4d1fc4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:09.446858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924f8e3-debd-403e-843b-0e9ba4d1fc4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:09.448269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924f8e3-debd-403e-843b-0e9ba4d1fc4f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:09.478635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db455b-926c-49a0-aefb-ed430ed44524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:09.479051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 243/488] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:09.479292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db455b-926c-49a0-aefb-ed430ed44524 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:09.479577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db455b-926c-49a0-aefb-ed430ed44524 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:09.484173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db455b-926c-49a0-aefb-ed430ed44524 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:09.485243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f8e3-debd-403e-843b-0e9ba4d1fc4f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:09.495996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db455b-926c-49a0-aefb-ed430ed44524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d6db455b-926c-49a0-aefb-ed430ed44524 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:09.501377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:09.502261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:09.503093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 246/489] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:09.513663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db455b-926c-49a0-aefb-ed430ed44524 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:09.554578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:09.560973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:09.619028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924f8e3-debd-403e-843b-0e9ba4d1fc4f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:09.657054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ce0e-a417-414f-81fa-b840b81f2490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:09.657995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ce0e-a417-414f-81fa-b840b81f2490 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:09.658451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ce0e-a417-414f-81fa-b840b81f2490 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:09.658949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ce0e-a417-414f-81fa-b840b81f2490 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:09.666842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832ce0e-a417-414f-81fa-b840b81f2490 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:09.668703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832ce0e-a417-414f-81fa-b840b81f2490 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0832ce0e-a417-414f-81fa-b840b81f2490 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:09.682999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:09.683988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:09.684902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:09.684902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:09.684902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:09.684902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:09.684902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:09.684902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:09.684902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:09.684902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:09.684902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:09.684902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:09.687314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 244/490] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Aug 6 01:05:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:09.699153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ce0e-a417-414f-81fa-b840b81f2490 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:09.709485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db455b-926c-49a0-aefb-ed430ed44524 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:09.761817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb3a87-9e02-4209-a7e5-0b697366fd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:09.762452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb3a87-9e02-4209-a7e5-0b697366fd86 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:09.762733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb3a87-9e02-4209-a7e5-0b697366fd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:09.763010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb3a87-9e02-4209-a7e5-0b697366fd86 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:09.763730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb3a87-9e02-4209-a7e5-0b697366fd86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:09.764068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb3a87-9e02-4209-a7e5-0b697366fd86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:09.765453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb3a87-9e02-4209-a7e5-0b697366fd86 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:09.786371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 247/491] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:09.787247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb3a87-9e02-4209-a7e5-0b697366fd86 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:09.860143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ce0e-a417-414f-81fa-b840b81f2490 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:09.862196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:09.898841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315df2c6-0694-4f00-985b-6dfe58f0cec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:09.899566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315df2c6-0694-4f00-985b-6dfe58f0cec0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:09.899959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315df2c6-0694-4f00-985b-6dfe58f0cec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:09.900306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315df2c6-0694-4f00-985b-6dfe58f0cec0 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:09.901222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315df2c6-0694-4f00-985b-6dfe58f0cec0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:09.901610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315df2c6-0694-4f00-985b-6dfe58f0cec0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:09.903160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315df2c6-0694-4f00-985b-6dfe58f0cec0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:09.908878 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:09.919469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb3a87-9e02-4209-a7e5-0b697366fd86 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:09.959340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 245/492] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Aug 6 01:05:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:09.960931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fe534-74c5-46ab-a891-ab59bf80a035 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:09.961693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fe534-74c5-46ab-a891-ab59bf80a035 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:09.962125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fe534-74c5-46ab-a891-ab59bf80a035 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:09.967716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 246/493] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:09.968727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fe534-74c5-46ab-a891-ab59bf80a035 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:09.981418 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26ac93-780c-48e7-a5f6-45ecf849bb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:09.982413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26ac93-780c-48e7-a5f6-45ecf849bb94 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:09.982774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26ac93-780c-48e7-a5f6-45ecf849bb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:09.983179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26ac93-780c-48e7-a5f6-45ecf849bb94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:09.998211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26ac93-780c-48e7-a5f6-45ecf849bb94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:10.000261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26ac93-780c-48e7-a5f6-45ecf849bb94 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-1a26ac93-780c-48e7-a5f6-45ecf849bb94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:10.018355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:10.019437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:10.020533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:10.020533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:10.020533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:10.020533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:10.021940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 248/494] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:10.023416 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca39a7-481f-4167-8f48-9994af208287 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.023981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca39a7-481f-4167-8f48-9994af208287 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.024473 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca39a7-481f-4167-8f48-9994af208287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.043497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-31ca39a7-481f-4167-8f48-9994af208287 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:10.044823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-31ca39a7-481f-4167-8f48-9994af208287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:10.094272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fe534-74c5-46ab-a891-ab59bf80a035 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:10.127807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 249/495] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:10.127807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca39a7-481f-4167-8f48-9994af208287 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:10.146108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44741a34-3628-4d05-b35c-b0c15272972a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:10.146819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44741a34-3628-4d05-b35c-b0c15272972a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.147166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44741a34-3628-4d05-b35c-b0c15272972a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.147635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44741a34-3628-4d05-b35c-b0c15272972a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.177551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:10.195232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:10.219598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44741a34-3628-4d05-b35c-b0c15272972a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:10.220862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca39a7-481f-4167-8f48-9994af208287 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:10.221164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44741a34-3628-4d05-b35c-b0c15272972a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:10.255339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1616e62-f735-49c4-b0d3-e9fb9f78614a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:10.255998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1616e62-f735-49c4-b0d3-e9fb9f78614a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.256282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1616e62-f735-49c4-b0d3-e9fb9f78614a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.256557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1616e62-f735-49c4-b0d3-e9fb9f78614a None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.257281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1616e62-f735-49c4-b0d3-e9fb9f78614a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:10.257582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1616e62-f735-49c4-b0d3-e9fb9f78614a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:10.258967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1616e62-f735-49c4-b0d3-e9fb9f78614a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:10.271000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 250/496] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:10.272349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f613d6-c691-4526-a917-a4944971f0fe None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.272669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f613d6-c691-4526-a917-a4944971f0fe None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.272944 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f613d6-c691-4526-a917-a4944971f0fe None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.274176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 251/497] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:10.274847 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f613d6-c691-4526-a917-a4944971f0fe None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:10.329457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 247/498] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:10.330828 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a947a-d38f-4393-aa5c-3216c792d5ba None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.331181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a947a-d38f-4393-aa5c-3216c792d5ba None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.331523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a947a-d38f-4393-aa5c-3216c792d5ba None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.335819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 248/499] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:10.335819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a947a-d38f-4393-aa5c-3216c792d5ba None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:10.380940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f613d6-c691-4526-a917-a4944971f0fe None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:10.393027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:10.412233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae67f09-5036-4ccb-9044-48fa29ec4df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:10.415220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae67f09-5036-4ccb-9044-48fa29ec4df9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.415220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae67f09-5036-4ccb-9044-48fa29ec4df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.415220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae67f09-5036-4ccb-9044-48fa29ec4df9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.415653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:10.424200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae67f09-5036-4ccb-9044-48fa29ec4df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0ae67f09-5036-4ccb-9044-48fa29ec4df9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:10.429623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:10.430564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:10.431363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 252/500] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:10.432060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c31cf6-748e-4695-a81b-f41bf75706dd None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.432133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c31cf6-748e-4695-a81b-f41bf75706dd None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.432409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c31cf6-748e-4695-a81b-f41bf75706dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.438963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a947a-d38f-4393-aa5c-3216c792d5ba None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:10.448991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e5c31cf6-748e-4695-a81b-f41bf75706dd None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:10.450157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e5c31cf6-748e-4695-a81b-f41bf75706dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:10.472713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7cb31-3571-4d24-8df6-2985c78472c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:10.473350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7cb31-3571-4d24-8df6-2985c78472c8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.473636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7cb31-3571-4d24-8df6-2985c78472c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.473913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7cb31-3571-4d24-8df6-2985c78472c8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.481748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7cb31-3571-4d24-8df6-2985c78472c8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:10.483174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7cb31-3571-4d24-8df6-2985c78472c8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-e2d7cb31-3571-4d24-8df6-2985c78472c8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:10.494231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:10.495131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:10.496061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:10.496061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:10.496061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:10.496061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:10.496061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:10.496061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:10.496061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:10.496061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:10.496061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:05:10.496061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:10.497029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 249/501] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:10.497854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7cb31-3571-4d24-8df6-2985c78472c8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:10.515526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 253/502] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:10.516818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879438b2-4cc3-4c9b-a665-c9bb2e5faf7c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.517107 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879438b2-4cc3-4c9b-a665-c9bb2e5faf7c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.517382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879438b2-4cc3-4c9b-a665-c9bb2e5faf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.543317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-879438b2-4cc3-4c9b-a665-c9bb2e5faf7c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:10.544546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-879438b2-4cc3-4c9b-a665-c9bb2e5faf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:10.596821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7cb31-3571-4d24-8df6-2985c78472c8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:10.611686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 254/503] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:10.612578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879438b2-4cc3-4c9b-a665-c9bb2e5faf7c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:10.616981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:10.631859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739bea3-9bc7-4253-b497-61306c0b4bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:10.632511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739bea3-9bc7-4253-b497-61306c0b4bee None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.632871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739bea3-9bc7-4253-b497-61306c0b4bee None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.633135 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739bea3-9bc7-4253-b497-61306c0b4bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.643371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739bea3-9bc7-4253-b497-61306c0b4bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:10.644939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739bea3-9bc7-4253-b497-61306c0b4bee None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-c739bea3-9bc7-4253-b497-61306c0b4bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:10.659415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:10.660583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:10.661450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:10.661450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:10.661450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:10.661450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:10.661450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 250/504] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:10.661725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739bea3-9bc7-4253-b497-61306c0b4bee None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:10.681831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:10.724388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879438b2-4cc3-4c9b-a665-c9bb2e5faf7c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:10.757063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407439fd-89f3-4100-aa41-7da1a074a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:10.757682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407439fd-89f3-4100-aa41-7da1a074a78d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.757955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407439fd-89f3-4100-aa41-7da1a074a78d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.758225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407439fd-89f3-4100-aa41-7da1a074a78d None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.758933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407439fd-89f3-4100-aa41-7da1a074a78d None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:10.759240 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407439fd-89f3-4100-aa41-7da1a074a78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:10.760657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407439fd-89f3-4100-aa41-7da1a074a78d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:10.762807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739bea3-9bc7-4253-b497-61306c0b4bee None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:10.768302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 255/505] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:10.769236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407439fd-89f3-4100-aa41-7da1a074a78d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:10.797979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10ba09-839b-4d93-a934-5bceb564d00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:10.798589 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10ba09-839b-4d93-a934-5bceb564d00c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.798870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10ba09-839b-4d93-a934-5bceb564d00c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.799143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10ba09-839b-4d93-a934-5bceb564d00c None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.799961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10ba09-839b-4d93-a934-5bceb564d00c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:10.800275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10ba09-839b-4d93-a934-5bceb564d00c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:10.801688 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10ba09-839b-4d93-a934-5bceb564d00c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:10.822998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 251/506] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Aug 6 01:05:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:10.822998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10ba09-839b-4d93-a934-5bceb564d00c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:10.850995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:10.866678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407439fd-89f3-4100-aa41-7da1a074a78d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:10.882776 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:10.900191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6520ba-ed1a-4c29-8ca9-2a92ca120c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:10.900832 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6520ba-ed1a-4c29-8ca9-2a92ca120c3d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.901113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6520ba-ed1a-4c29-8ca9-2a92ca120c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.901385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6520ba-ed1a-4c29-8ca9-2a92ca120c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.912115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6520ba-ed1a-4c29-8ca9-2a92ca120c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:10.913657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6520ba-ed1a-4c29-8ca9-2a92ca120c3d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:10.918015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10ba09-839b-4d93-a934-5bceb564d00c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-1b6520ba-ed1a-4c29-8ca9-2a92ca120c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:10.927142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:10.928200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:10.929255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:10.929255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:10.929255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:10.929255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:10.929255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 256/507] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:10.929592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6520ba-ed1a-4c29-8ca9-2a92ca120c3d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:10.954645 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92c697-4d7e-4fd7-b512-9338d6388515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:10.955378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92c697-4d7e-4fd7-b512-9338d6388515 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:10.955722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92c697-4d7e-4fd7-b512-9338d6388515 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:10.956038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92c697-4d7e-4fd7-b512-9338d6388515 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:10.966699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92c697-4d7e-4fd7-b512-9338d6388515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2b92c697-4d7e-4fd7-b512-9338d6388515 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:10.971473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:10.972522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:10.973561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 252/508] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:10.974416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92c697-4d7e-4fd7-b512-9338d6388515 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.029952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6520ba-ed1a-4c29-8ca9-2a92ca120c3d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.062127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:11.063503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410da26-e17d-43fc-a5e2-b5eb11596caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.064159 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410da26-e17d-43fc-a5e2-b5eb11596caf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.064437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410da26-e17d-43fc-a5e2-b5eb11596caf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.064707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410da26-e17d-43fc-a5e2-b5eb11596caf None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.065386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410da26-e17d-43fc-a5e2-b5eb11596caf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:11.065689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410da26-e17d-43fc-a5e2-b5eb11596caf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.067055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410da26-e17d-43fc-a5e2-b5eb11596caf None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.071379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92c697-4d7e-4fd7-b512-9338d6388515 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.078671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 257/509] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:11.079530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410da26-e17d-43fc-a5e2-b5eb11596caf None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.106261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6792d5-6878-43fd-bf9a-7f6e6fae217a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.106899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6792d5-6878-43fd-bf9a-7f6e6fae217a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.107184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6792d5-6878-43fd-bf9a-7f6e6fae217a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.107461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6792d5-6878-43fd-bf9a-7f6e6fae217a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.107839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:11.118065 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6792d5-6878-43fd-bf9a-7f6e6fae217a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.119540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6792d5-6878-43fd-bf9a-7f6e6fae217a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-ea6792d5-6878-43fd-bf9a-7f6e6fae217a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:11.134071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:11.135066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:11.136172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:11.136172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:11.136172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:11.136172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:11.136172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 253/510] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:11.140417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6792d5-6878-43fd-bf9a-7f6e6fae217a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.180068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410da26-e17d-43fc-a5e2-b5eb11596caf None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.213048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b869b3e-faf0-46ac-9a2f-45b139ed1ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.213599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b869b3e-faf0-46ac-9a2f-45b139ed1ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.213878 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b869b3e-faf0-46ac-9a2f-45b139ed1ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.214150 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b869b3e-faf0-46ac-9a2f-45b139ed1ad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.239977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6792d5-6878-43fd-bf9a-7f6e6fae217a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.272333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c09d08-df56-4e88-8532-e8780cbf9700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.273463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c09d08-df56-4e88-8532-e8780cbf9700 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.273743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c09d08-df56-4e88-8532-e8780cbf9700 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.274017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c09d08-df56-4e88-8532-e8780cbf9700 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.274701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c09d08-df56-4e88-8532-e8780cbf9700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:11.275005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c09d08-df56-4e88-8532-e8780cbf9700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.276408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c09d08-df56-4e88-8532-e8780cbf9700 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.278692 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b869b3e-faf0-46ac-9a2f-45b139ed1ad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.280176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b869b3e-faf0-46ac-9a2f-45b139ed1ad6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.287584 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 254/511] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:11.288972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c09d08-df56-4e88-8532-e8780cbf9700 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.291713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:11.311820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:11.380241 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 258/512] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:11.381264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b869b3e-faf0-46ac-9a2f-45b139ed1ad6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.388435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c09d08-df56-4e88-8532-e8780cbf9700 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.422789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc1e9e-43e4-43a7-b0a0-a88f8d3eba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.423390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc1e9e-43e4-43a7-b0a0-a88f8d3eba35 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.423679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc1e9e-43e4-43a7-b0a0-a88f8d3eba35 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.423983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc1e9e-43e4-43a7-b0a0-a88f8d3eba35 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.424705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc1e9e-43e4-43a7-b0a0-a88f8d3eba35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:11.425014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc1e9e-43e4-43a7-b0a0-a88f8d3eba35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.426387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc1e9e-43e4-43a7-b0a0-a88f8d3eba35 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.434051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 255/513] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:11.434823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc1e9e-43e4-43a7-b0a0-a88f8d3eba35 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.482105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b869b3e-faf0-46ac-9a2f-45b139ed1ad6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.495532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:11.507997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:11.515331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff670dda-d466-45b6-b0bb-c4c8fde2eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.516001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff670dda-d466-45b6-b0bb-c4c8fde2eafd None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.516284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff670dda-d466-45b6-b0bb-c4c8fde2eafd None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.516557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff670dda-d466-45b6-b0bb-c4c8fde2eafd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff670dda-d466-45b6-b0bb-c4c8fde2eafd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.524976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff670dda-d466-45b6-b0bb-c4c8fde2eafd None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.532955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc1e9e-43e4-43a7-b0a0-a88f8d3eba35 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ff670dda-d466-45b6-b0bb-c4c8fde2eafd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:11.536481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:11.537456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:11.538865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:11.538865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:11.538865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:11.538865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:11.538865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:11.538865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:11.538865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:11.538865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:11.538865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 259/514] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:11.539504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff670dda-d466-45b6-b0bb-c4c8fde2eafd None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.569110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ff8e6-d02b-4d90-950c-91385d1f04a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.570268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ff8e6-d02b-4d90-950c-91385d1f04a5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.570545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ff8e6-d02b-4d90-950c-91385d1f04a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.570821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ff8e6-d02b-4d90-950c-91385d1f04a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.571502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ff8e6-d02b-4d90-950c-91385d1f04a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:11.571835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ff8e6-d02b-4d90-950c-91385d1f04a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.573207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ff8e6-d02b-4d90-950c-91385d1f04a5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.587640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 256/515] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:11.588616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ff8e6-d02b-4d90-950c-91385d1f04a5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.645948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff670dda-d466-45b6-b0bb-c4c8fde2eafd None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.678718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643490d-2ac4-4f35-b1d2-a2e9f63cfff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.679337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643490d-2ac4-4f35-b1d2-a2e9f63cfff2 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.679622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643490d-2ac4-4f35-b1d2-a2e9f63cfff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.679937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643490d-2ac4-4f35-b1d2-a2e9f63cfff2 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.680617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643490d-2ac4-4f35-b1d2-a2e9f63cfff2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.682067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643490d-2ac4-4f35-b1d2-a2e9f63cfff2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.692571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ff8e6-d02b-4d90-950c-91385d1f04a5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.716245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:11.726232 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88097eb4-b89a-4adf-bb81-594a2934983a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.726861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88097eb4-b89a-4adf-bb81-594a2934983a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.727143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88097eb4-b89a-4adf-bb81-594a2934983a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.727415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88097eb4-b89a-4adf-bb81-594a2934983a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.736998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88097eb4-b89a-4adf-bb81-594a2934983a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.738368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88097eb4-b89a-4adf-bb81-594a2934983a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-88097eb4-b89a-4adf-bb81-594a2934983a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:11.752182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:11.762215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:11.763281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:11.763281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:11.763281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:11.763281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:11.763281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 257/516] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:11.763281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88097eb4-b89a-4adf-bb81-594a2934983a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.781843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 260/517] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:05:11 2022] POST /identity/v3/users => generated 247 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:11.782676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643490d-2ac4-4f35-b1d2-a2e9f63cfff2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.833114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:11.874149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88097eb4-b89a-4adf-bb81-594a2934983a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.891667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643490d-2ac4-4f35-b1d2-a2e9f63cfff2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:11.913466 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d76841-3578-4442-9f01-85ec93c21393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.914114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76841-3578-4442-9f01-85ec93c21393 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.914395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76841-3578-4442-9f01-85ec93c21393 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.914670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76841-3578-4442-9f01-85ec93c21393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.928265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d76841-3578-4442-9f01-85ec93c21393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.930292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d76841-3578-4442-9f01-85ec93c21393 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.937561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85a796-ba96-44d6-b947-a8f777dfd1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:11.938203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85a796-ba96-44d6-b947-a8f777dfd1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:11.938496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85a796-ba96-44d6-b947-a8f777dfd1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:11.938782 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85a796-ba96-44d6-b947-a8f777dfd1f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:11.939822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85a796-ba96-44d6-b947-a8f777dfd1f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:11.939933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85a796-ba96-44d6-b947-a8f777dfd1f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:11.941322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85a796-ba96-44d6-b947-a8f777dfd1f0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-60d76841-3578-4442-9f01-85ec93c21393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:11.946406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:11.947281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:11.948218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:11.948218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:11.948218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:11.948218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:11.948218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 258/518] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:11.948920 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d76841-3578-4442-9f01-85ec93c21393 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.961238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 261/519] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:11.961238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85a796-ba96-44d6-b947-a8f777dfd1f0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:11.984224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:12.062392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d76841-3578-4442-9f01-85ec93c21393 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:12.074556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85a796-ba96-44d6-b947-a8f777dfd1f0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:12.078245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:12.096646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4089df-fefe-4184-8d0c-afb7f3dd65b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:12.097287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4089df-fefe-4184-8d0c-afb7f3dd65b7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:12.097575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4089df-fefe-4184-8d0c-afb7f3dd65b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:12.097856 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4089df-fefe-4184-8d0c-afb7f3dd65b7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:12.105011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4089df-fefe-4184-8d0c-afb7f3dd65b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:12.106483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4089df-fefe-4184-8d0c-afb7f3dd65b7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:12.111501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e22991-ca8d-4d8e-9322-ffd6da0bdcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:12.112156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e22991-ca8d-4d8e-9322-ffd6da0bdcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:12.112440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e22991-ca8d-4d8e-9322-ffd6da0bdcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:12.112715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e22991-ca8d-4d8e-9322-ffd6da0bdcb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:12.113405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e22991-ca8d-4d8e-9322-ffd6da0bdcb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:12.113712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e22991-ca8d-4d8e-9322-ffd6da0bdcb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:12.115086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e22991-ca8d-4d8e-9322-ffd6da0bdcb7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-5f4089df-fefe-4184-8d0c-afb7f3dd65b7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:12.121145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:12.122017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:12.122908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:12.122908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:12.122908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:12.122908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:12.122908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:12.122908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:12.122908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:12.122908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:12.122908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:05:12.122908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:12.123411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 259/520] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:12.124071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4089df-fefe-4184-8d0c-afb7f3dd65b7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:12.131315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 262/521] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:12.133004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e22991-ca8d-4d8e-9322-ffd6da0bdcb7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:12.220037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:12.235182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4089df-fefe-4184-8d0c-afb7f3dd65b7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:12.241643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e22991-ca8d-4d8e-9322-ffd6da0bdcb7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:12.276539 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01be523-037b-459d-b258-222caca0db3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:12.276739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388be30-96b4-4772-92d5-d7961031f354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:12.277165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01be523-037b-459d-b258-222caca0db3c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:12.277280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388be30-96b4-4772-92d5-d7961031f354 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:12.277439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01be523-037b-459d-b258-222caca0db3c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:12.277533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388be30-96b4-4772-92d5-d7961031f354 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:12.277716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01be523-037b-459d-b258-222caca0db3c None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:12.277812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388be30-96b4-4772-92d5-d7961031f354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:12.278411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01be523-037b-459d-b258-222caca0db3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:12.278584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388be30-96b4-4772-92d5-d7961031f354 None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=db1b205d165543d0a3a69a2db6af5d1c) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:12.278746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01be523-037b-459d-b258-222caca0db3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:12.278845 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388be30-96b4-4772-92d5-d7961031f354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:12.280140 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01be523-037b-459d-b258-222caca0db3c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:12.280303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388be30-96b4-4772-92d5-d7961031f354 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:12.291887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 263/522] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:12 2022] GET /identity/v3/role_assignments?user.id=db1b205d165543d0a3a69a2db6af5d1c&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:12.291887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388be30-96b4-4772-92d5-d7961031f354 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:12.293338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 260/523] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:12.294214 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01be523-037b-459d-b258-222caca0db3c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:12.333838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:12.417472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01be523-037b-459d-b258-222caca0db3c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:12.425533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388be30-96b4-4772-92d5-d7961031f354 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:12.462045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77497e-9c5b-4831-a5bd-466da7eedfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:12.462987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77497e-9c5b-4831-a5bd-466da7eedfc8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:12.463273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77497e-9c5b-4831-a5bd-466da7eedfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:12.463806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77497e-9c5b-4831-a5bd-466da7eedfc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:12.469468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1962ae5-7821-4147-9d91-22c53fd41e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:12.470095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1962ae5-7821-4147-9d91-22c53fd41e49 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:12.470374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1962ae5-7821-4147-9d91-22c53fd41e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:12.470646 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1962ae5-7821-4147-9d91-22c53fd41e49 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:12.471444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1962ae5-7821-4147-9d91-22c53fd41e49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:12.471753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1962ae5-7821-4147-9d91-22c53fd41e49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:12.472078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f77497e-9c5b-4831-a5bd-466da7eedfc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:12.473220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1962ae5-7821-4147-9d91-22c53fd41e49 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:12.473545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f77497e-9c5b-4831-a5bd-466da7eedfc8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-1f77497e-9c5b-4831-a5bd-466da7eedfc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:12.486608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:12.487608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:12.488491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:12.488491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:12.488491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:12.488491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:12.488491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:12.488491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:12.488491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:12.488491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:12.488491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 261/524] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:12.488958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77497e-9c5b-4831-a5bd-466da7eedfc8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:12.504782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:12.521637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 264/525] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Aug 6 01:05:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:12.521637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1962ae5-7821-4147-9d91-22c53fd41e49 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:12.616331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77497e-9c5b-4831-a5bd-466da7eedfc8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:12.636108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:12.652208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1962ae5-7821-4147-9d91-22c53fd41e49 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:12.658066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3d88c-17d5-48c8-a47b-96a854dc9c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:12.658938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3d88c-17d5-48c8-a47b-96a854dc9c1a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:12.659399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3d88c-17d5-48c8-a47b-96a854dc9c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:12.660305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3d88c-17d5-48c8-a47b-96a854dc9c1a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:12.671939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3d88c-17d5-48c8-a47b-96a854dc9c1a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:12.673503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3d88c-17d5-48c8-a47b-96a854dc9c1a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-76c3d88c-17d5-48c8-a47b-96a854dc9c1a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:12.685984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:12.686916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:12.687913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:12.687913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:12.687913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:12.687913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:12.687913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:12.687913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:12.687913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:12.687913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:12.687913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:05:12.687913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:12.691945 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 262/526] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:12.692844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3d88c-17d5-48c8-a47b-96a854dc9c1a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:12.695305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5d55b-6dd1-4951-915b-5f0d52950304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:12.696157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d55b-6dd1-4951-915b-5f0d52950304 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:12.696780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d55b-6dd1-4951-915b-5f0d52950304 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:12.697231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d55b-6dd1-4951-915b-5f0d52950304 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:12.698949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5d55b-6dd1-4951-915b-5f0d52950304 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:12.699440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5d55b-6dd1-4951-915b-5f0d52950304 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:12.701091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5d55b-6dd1-4951-915b-5f0d52950304 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:12.721136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 265/527] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:12.721136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5d55b-6dd1-4951-915b-5f0d52950304 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:12.780507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:12.827666 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3d88c-17d5-48c8-a47b-96a854dc9c1a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:12.837265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5d55b-6dd1-4951-915b-5f0d52950304 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:12.875561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5686a-67a0-4bfc-bb34-d4ed68806923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:12.876337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5686a-67a0-4bfc-bb34-d4ed68806923 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:12.876692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5686a-67a0-4bfc-bb34-d4ed68806923 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:12.877137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5686a-67a0-4bfc-bb34-d4ed68806923 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:12.879242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad070de-eec0-4469-93e7-b951a8c56b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:12.880081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad070de-eec0-4469-93e7-b951a8c56b3f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:12.880619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad070de-eec0-4469-93e7-b951a8c56b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:12.881099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad070de-eec0-4469-93e7-b951a8c56b3f None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:12.882734 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad070de-eec0-4469-93e7-b951a8c56b3f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:12.883245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad070de-eec0-4469-93e7-b951a8c56b3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:12.884911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad070de-eec0-4469-93e7-b951a8c56b3f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:12.906943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 266/528] 10.222.0.25 () {56 vars in 1139 bytes} [Sat Aug 6 01:05:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:12.908403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad070de-eec0-4469-93e7-b951a8c56b3f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:12.913726 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:12.953246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5686a-67a0-4bfc-bb34-d4ed68806923 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:12.956073 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5686a-67a0-4bfc-bb34-d4ed68806923 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:13.015503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad070de-eec0-4469-93e7-b951a8c56b3f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:13.029464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:13.052303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f297a-d7ff-4406-9e9b-e226bd5d1b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:13.052971 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f297a-d7ff-4406-9e9b-e226bd5d1b3a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:13.053366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f297a-d7ff-4406-9e9b-e226bd5d1b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:13.053783 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f297a-d7ff-4406-9e9b-e226bd5d1b3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:13.055217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f297a-d7ff-4406-9e9b-e226bd5d1b3a None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=ddd0e588d52e40f08a2dc1062ecd4232) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:13.055595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f297a-d7ff-4406-9e9b-e226bd5d1b3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:13.057184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f297a-d7ff-4406-9e9b-e226bd5d1b3a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:13.067841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 267/529] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:12 2022] GET /identity/v3/role_assignments?user.id=ddd0e588d52e40f08a2dc1062ecd4232&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:13.067841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f297a-d7ff-4406-9e9b-e226bd5d1b3a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:13.086120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 263/530] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:13.091938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5686a-67a0-4bfc-bb34-d4ed68806923 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:13.215668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:13.274908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f297a-d7ff-4406-9e9b-e226bd5d1b3a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:13.280850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5686a-67a0-4bfc-bb34-d4ed68806923 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:13.319011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a444c-4a35-47c2-b2f6-5b4eac4f0612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:13.320234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a444c-4a35-47c2-b2f6-5b4eac4f0612 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:13.320719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a444c-4a35-47c2-b2f6-5b4eac4f0612 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:13.321188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a444c-4a35-47c2-b2f6-5b4eac4f0612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:13.335207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392ef55-1b3c-4062-bb4d-80aa78d1be3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:13.335890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392ef55-1b3c-4062-bb4d-80aa78d1be3c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:13.336173 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392ef55-1b3c-4062-bb4d-80aa78d1be3c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:13.336495 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392ef55-1b3c-4062-bb4d-80aa78d1be3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:13.358969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392ef55-1b3c-4062-bb4d-80aa78d1be3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:13.361186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392ef55-1b3c-4062-bb4d-80aa78d1be3c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0392ef55-1b3c-4062-bb4d-80aa78d1be3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:13.388886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:13.389818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:13.390690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:13.390690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:13.390690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:13.390690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:13.399292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 264/531] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:13.406223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392ef55-1b3c-4062-bb4d-80aa78d1be3c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:13.454296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:13.471943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a444c-4a35-47c2-b2f6-5b4eac4f0612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:13.471943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a444c-4a35-47c2-b2f6-5b4eac4f0612 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:13.596209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:13.611572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392ef55-1b3c-4062-bb4d-80aa78d1be3c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:13.653304 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 268/532] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:13.658848 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a444c-4a35-47c2-b2f6-5b4eac4f0612 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:13.685210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c8f1d-4907-4048-9693-6d8d47135604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:13.685927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c8f1d-4907-4048-9693-6d8d47135604 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:13.686215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c8f1d-4907-4048-9693-6d8d47135604 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:13.686489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c8f1d-4907-4048-9693-6d8d47135604 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:13.687184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c8f1d-4907-4048-9693-6d8d47135604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:13.687581 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c8f1d-4907-4048-9693-6d8d47135604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:13.689065 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c8f1d-4907-4048-9693-6d8d47135604 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:13.717912 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 265/533] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:13.724790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c8f1d-4907-4048-9693-6d8d47135604 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:13.762218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a444c-4a35-47c2-b2f6-5b4eac4f0612 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:13.797953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e908d4e-a05c-47fd-8fae-40227d6f6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:13.798641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e908d4e-a05c-47fd-8fae-40227d6f6f52 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:13.799018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e908d4e-a05c-47fd-8fae-40227d6f6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:13.799329 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e908d4e-a05c-47fd-8fae-40227d6f6f52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:13.810191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e908d4e-a05c-47fd-8fae-40227d6f6f52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:13.811659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e908d4e-a05c-47fd-8fae-40227d6f6f52 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-2e908d4e-a05c-47fd-8fae-40227d6f6f52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:13.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:13.828438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:13.829296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:13.829296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:13.829296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:13.829296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:13.829296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 269/534] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:13.834723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e908d4e-a05c-47fd-8fae-40227d6f6f52 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:13.872201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:13.943084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e908d4e-a05c-47fd-8fae-40227d6f6f52 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:13.949993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:13.980100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c8f1d-4907-4048-9693-6d8d47135604 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:14.000596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5675c35-ff82-4d5c-8567-e7fa544f564b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:14.001243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5675c35-ff82-4d5c-8567-e7fa544f564b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.001527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5675c35-ff82-4d5c-8567-e7fa544f564b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.001806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5675c35-ff82-4d5c-8567-e7fa544f564b None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.002504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5675c35-ff82-4d5c-8567-e7fa544f564b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:14.002811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5675c35-ff82-4d5c-8567-e7fa544f564b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:14.004346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5675c35-ff82-4d5c-8567-e7fa544f564b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:14.015500 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff73fdb-bf0f-4572-8229-988e7b8cb685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:14.016178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff73fdb-bf0f-4572-8229-988e7b8cb685 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.016461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff73fdb-bf0f-4572-8229-988e7b8cb685 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.016739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff73fdb-bf0f-4572-8229-988e7b8cb685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.024357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff73fdb-bf0f-4572-8229-988e7b8cb685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:14.025156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 270/535] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:14.025826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff73fdb-bf0f-4572-8229-988e7b8cb685 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:14.026475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87dfa4-84b6-4d39-9b6b-3bfde47d3f54 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.026766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87dfa4-84b6-4d39-9b6b-3bfde47d3f54 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.027042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87dfa4-84b6-4d39-9b6b-3bfde47d3f54 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.028640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 271/536] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:14.032615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87dfa4-84b6-4d39-9b6b-3bfde47d3f54 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-dff73fdb-bf0f-4572-8229-988e7b8cb685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:14.037242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:14.039226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:14.041531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:14.041531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:14.041531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:14.041531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:14.041531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:14.041531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:14.041531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:14.041531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:14.041531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 266/537] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:14.042639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7997b1-6826-43d1-81b1-bd8b6a9468b4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.046793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7997b1-6826-43d1-81b1-bd8b6a9468b4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.046793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7997b1-6826-43d1-81b1-bd8b6a9468b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.062141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8a7997b1-6826-43d1-81b1-bd8b6a9468b4 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:14.063354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8a7997b1-6826-43d1-81b1-bd8b6a9468b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:14.134440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 267/538] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:14.135345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7997b1-6826-43d1-81b1-bd8b6a9468b4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:14.158191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87dfa4-84b6-4d39-9b6b-3bfde47d3f54 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:14.160102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:14.205702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:14.208631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bfbf2-d402-4342-ba39-468a87f97a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:14.209490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bfbf2-d402-4342-ba39-468a87f97a45 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.209939 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bfbf2-d402-4342-ba39-468a87f97a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.210352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bfbf2-d402-4342-ba39-468a87f97a45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.221927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bfbf2-d402-4342-ba39-468a87f97a45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:14.224072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bfbf2-d402-4342-ba39-468a87f97a45 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:14.237192 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7997b1-6826-43d1-81b1-bd8b6a9468b4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-315bfbf2-d402-4342-ba39-468a87f97a45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:14.240895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:14.242307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:14.243140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:14.243140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:14.243140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:14.243140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:14.243140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:14.243140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:14.243140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:14.243140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:14.243140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 272/539] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:14.243596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bfbf2-d402-4342-ba39-468a87f97a45 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:14.288091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79988bc9-9d5f-4feb-ad8a-f6ffd6884f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:14.288715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79988bc9-9d5f-4feb-ad8a-f6ffd6884f06 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.288996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79988bc9-9d5f-4feb-ad8a-f6ffd6884f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.289271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79988bc9-9d5f-4feb-ad8a-f6ffd6884f06 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.290030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79988bc9-9d5f-4feb-ad8a-f6ffd6884f06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:14.290341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79988bc9-9d5f-4feb-ad8a-f6ffd6884f06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:14.291818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79988bc9-9d5f-4feb-ad8a-f6ffd6884f06 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:14.314808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 268/540] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:14.315667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79988bc9-9d5f-4feb-ad8a-f6ffd6884f06 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:14.375852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bfbf2-d402-4342-ba39-468a87f97a45 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:14.413009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478bf652-0547-4dc5-a01e-5bf09c16f57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:14.413715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bf652-0547-4dc5-a01e-5bf09c16f57a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.413997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bf652-0547-4dc5-a01e-5bf09c16f57a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.414273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bf652-0547-4dc5-a01e-5bf09c16f57a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.422936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:14.425725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478bf652-0547-4dc5-a01e-5bf09c16f57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-478bf652-0547-4dc5-a01e-5bf09c16f57a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:14.431675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:14.432683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 273/541] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:14.438256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478bf652-0547-4dc5-a01e-5bf09c16f57a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:14.448206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79988bc9-9d5f-4feb-ad8a-f6ffd6884f06 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:14.482556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:14.488549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6ed7a-6a33-4947-bf00-2ff63e2637a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:14.489109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ed7a-6a33-4947-bf00-2ff63e2637a6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.489415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ed7a-6a33-4947-bf00-2ff63e2637a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.489670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6ed7a-6a33-4947-bf00-2ff63e2637a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.490401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6ed7a-6a33-4947-bf00-2ff63e2637a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:14.490701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6ed7a-6a33-4947-bf00-2ff63e2637a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:14.492853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6ed7a-6a33-4947-bf00-2ff63e2637a6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:14.504894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 269/542] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:14.506141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08aefa-62b8-4176-b60c-bbebac40b331 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.506428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08aefa-62b8-4176-b60c-bbebac40b331 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.506701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08aefa-62b8-4176-b60c-bbebac40b331 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.508664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 270/543] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:14.513538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08aefa-62b8-4176-b60c-bbebac40b331 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:14.543278 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478bf652-0547-4dc5-a01e-5bf09c16f57a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:14.587878 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8389452-4cdd-4c0e-8230-5870d7ba8c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:14.588508 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8389452-4cdd-4c0e-8230-5870d7ba8c5a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.588860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8389452-4cdd-4c0e-8230-5870d7ba8c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.589320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8389452-4cdd-4c0e-8230-5870d7ba8c5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.590748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8389452-4cdd-4c0e-8230-5870d7ba8c5a None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=9a234a96eccd4f38b0213d7d0461ad24) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:14.591308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8389452-4cdd-4c0e-8230-5870d7ba8c5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:14.593536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8389452-4cdd-4c0e-8230-5870d7ba8c5a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:14.604110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 274/544] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:14 2022] GET /identity/v3/role_assignments?user.id=9a234a96eccd4f38b0213d7d0461ad24&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:14.605950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8389452-4cdd-4c0e-8230-5870d7ba8c5a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:14.640955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08aefa-62b8-4176-b60c-bbebac40b331 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:14.693410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:14.697512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f1d8c-e785-4e2b-a869-09adb70a0f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:14.698363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f1d8c-e785-4e2b-a869-09adb70a0f7f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.698707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f1d8c-e785-4e2b-a869-09adb70a0f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.698986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f1d8c-e785-4e2b-a869-09adb70a0f7f None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.699774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f1d8c-e785-4e2b-a869-09adb70a0f7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:14.700749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f1d8c-e785-4e2b-a869-09adb70a0f7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:14.702853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f1d8c-e785-4e2b-a869-09adb70a0f7f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:14.719949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 271/545] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:14.721946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c5241-ae67-47cf-817a-164e4d83c82e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.723414 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c5241-ae67-47cf-817a-164e4d83c82e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.724747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c5241-ae67-47cf-817a-164e4d83c82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.745438 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-085c5241-ae67-47cf-817a-164e4d83c82e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:14.746562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-085c5241-ae67-47cf-817a-164e4d83c82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:14.755484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8389452-4cdd-4c0e-8230-5870d7ba8c5a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:14.809150 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3af331-82ad-4727-9509-45fa24a660dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:14.812906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:14.814442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3af331-82ad-4727-9509-45fa24a660dc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:14.815581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3af331-82ad-4727-9509-45fa24a660dc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:14.816307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3af331-82ad-4727-9509-45fa24a660dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:14.817450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3af331-82ad-4727-9509-45fa24a660dc None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=2c14fb133a0f419390fb0d00c600bdd4) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:14.817980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3af331-82ad-4727-9509-45fa24a660dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:14.819754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3af331-82ad-4727-9509-45fa24a660dc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:14.828866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 275/546] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:14 2022] GET /identity/v3/role_assignments?user.id=2c14fb133a0f419390fb0d00c600bdd4&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:14.830067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3af331-82ad-4727-9509-45fa24a660dc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:14.843920 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 272/547] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:14.872386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c5241-ae67-47cf-817a-164e4d83c82e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:14.985534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3af331-82ad-4727-9509-45fa24a660dc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:15.048254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:15.065861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731cec1-e938-48b9-9011-76bb895e2ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:15.066529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731cec1-e938-48b9-9011-76bb895e2ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:15.066819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731cec1-e938-48b9-9011-76bb895e2ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:15.067095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731cec1-e938-48b9-9011-76bb895e2ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:15.079163 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c5241-ae67-47cf-817a-164e4d83c82e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:15.087453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731cec1-e938-48b9-9011-76bb895e2ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:15.089086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731cec1-e938-48b9-9011-76bb895e2ec5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d731cec1-e938-48b9-9011-76bb895e2ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:15.111321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:15.112262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:15.113211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:15.113211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:15.113211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:15.113211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:15.113211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 276/548] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:15.119242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731cec1-e938-48b9-9011-76bb895e2ec5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:15.137582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f6a70-23d9-4dc0-90ce-211433e6a3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:15.138240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6a70-23d9-4dc0-90ce-211433e6a3fc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:15.138517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6a70-23d9-4dc0-90ce-211433e6a3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:15.138792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6a70-23d9-4dc0-90ce-211433e6a3fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:15.160831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f6a70-23d9-4dc0-90ce-211433e6a3fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-4f6f6a70-23d9-4dc0-90ce-211433e6a3fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:15.167454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:15.169009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:15.169910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 273/549] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:15.182443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f6a70-23d9-4dc0-90ce-211433e6a3fc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:15.239375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:15.302494 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f6a70-23d9-4dc0-90ce-211433e6a3fc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:15.323736 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731cec1-e938-48b9-9011-76bb895e2ec5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:15.341720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8f4d8-05c1-4529-98a0-84998e534ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:15.342828 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8f4d8-05c1-4529-98a0-84998e534ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:15.344964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8f4d8-05c1-4529-98a0-84998e534ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:15.345393 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8f4d8-05c1-4529-98a0-84998e534ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:15.346267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8f4d8-05c1-4529-98a0-84998e534ee8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:15.347607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8f4d8-05c1-4529-98a0-84998e534ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:15.350032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8f4d8-05c1-4529-98a0-84998e534ee8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:15.368093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 274/550] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:15.376569 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8f4d8-05c1-4529-98a0-84998e534ee8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:15.391238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5089cb2-8b52-4fcf-b9fc-c17ec300d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:15.392129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5089cb2-8b52-4fcf-b9fc-c17ec300d3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:15.392588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5089cb2-8b52-4fcf-b9fc-c17ec300d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:15.393010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5089cb2-8b52-4fcf-b9fc-c17ec300d3e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:15.393856 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5089cb2-8b52-4fcf-b9fc-c17ec300d3e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:15.396733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5089cb2-8b52-4fcf-b9fc-c17ec300d3e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:15.401156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5089cb2-8b52-4fcf-b9fc-c17ec300d3e3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:15.423833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 277/551] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:15.429323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5089cb2-8b52-4fcf-b9fc-c17ec300d3e3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:15.482489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:15.551727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8f4d8-05c1-4529-98a0-84998e534ee8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:15.554329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:15.556940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5089cb2-8b52-4fcf-b9fc-c17ec300d3e3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:15.590465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e33fa-16ac-488f-b33d-34581afde492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:15.591118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e33fa-16ac-488f-b33d-34581afde492 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:15.591405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e33fa-16ac-488f-b33d-34581afde492 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:15.591685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e33fa-16ac-488f-b33d-34581afde492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:15.599397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825357fe-7f68-4b5b-bba0-efc5cafd2b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:15.599397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825357fe-7f68-4b5b-bba0-efc5cafd2b74 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:15.599397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825357fe-7f68-4b5b-bba0-efc5cafd2b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:15.599397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825357fe-7f68-4b5b-bba0-efc5cafd2b74 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:15.603345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e33fa-16ac-488f-b33d-34581afde492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:15.604556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825357fe-7f68-4b5b-bba0-efc5cafd2b74 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:15.604971 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e33fa-16ac-488f-b33d-34581afde492 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:15.606064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825357fe-7f68-4b5b-bba0-efc5cafd2b74 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2c2e33fa-16ac-488f-b33d-34581afde492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:15.620871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:15.621772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:15.622642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:15.622642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:15.622642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:15.622642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-825357fe-7f68-4b5b-bba0-efc5cafd2b74 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:15.622883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:15.623716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:15.624781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:15.624781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:15.624781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:15.624781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:15.624781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:15.624781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:15.624781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:15.624781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:15.624781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:15.624781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:15.625312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 275/552] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:15.625312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065c1c2-70a9-41db-ad57-b7dae972fc87 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:15.625312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065c1c2-70a9-41db-ad57-b7dae972fc87 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:15.625312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065c1c2-70a9-41db-ad57-b7dae972fc87 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:15.626346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 276/553] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:15.629105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065c1c2-70a9-41db-ad57-b7dae972fc87 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:15.629288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 278/554] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:15.634969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825357fe-7f68-4b5b-bba0-efc5cafd2b74 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:15.748939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065c1c2-70a9-41db-ad57-b7dae972fc87 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:15.759934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:15.764575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825357fe-7f68-4b5b-bba0-efc5cafd2b74 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:15.785692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951bc00-f06b-46bd-b11d-45cfbfac3be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:15.786469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951bc00-f06b-46bd-b11d-45cfbfac3be3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:15.787335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951bc00-f06b-46bd-b11d-45cfbfac3be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:15.787932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951bc00-f06b-46bd-b11d-45cfbfac3be3 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:15.789258 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951bc00-f06b-46bd-b11d-45cfbfac3be3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:15.789642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951bc00-f06b-46bd-b11d-45cfbfac3be3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:15.791090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951bc00-f06b-46bd-b11d-45cfbfac3be3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:15.811017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 277/555] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:15.811017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa87dc-b04b-42a0-9ec1-006dfdbab1b4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:15.811017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa87dc-b04b-42a0-9ec1-006dfdbab1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:15.811017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa87dc-b04b-42a0-9ec1-006dfdbab1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:15.814253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0bd4-a223-482d-b3e9-4bab97813b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:15.815139 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0bd4-a223-482d-b3e9-4bab97813b9d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:15.815615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0bd4-a223-482d-b3e9-4bab97813b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:15.816393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0bd4-a223-482d-b3e9-4bab97813b9d None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:15.818341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0bd4-a223-482d-b3e9-4bab97813b9d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:15.818837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0bd4-a223-482d-b3e9-4bab97813b9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:15.820572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0bd4-a223-482d-b3e9-4bab97813b9d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:15.829700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7afa87dc-b04b-42a0-9ec1-006dfdbab1b4 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:15.831160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7afa87dc-b04b-42a0-9ec1-006dfdbab1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:15.847711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 279/556] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Aug 6 01:05:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:15.848407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0bd4-a223-482d-b3e9-4bab97813b9d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:15.850031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:15.930443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 278/557] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:15.931652 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa87dc-b04b-42a0-9ec1-006dfdbab1b4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:15.989979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0bd4-a223-482d-b3e9-4bab97813b9d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:16.034022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad872d0d-786d-493c-8f29-f521d7fcb2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:16.034675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad872d0d-786d-493c-8f29-f521d7fcb2df None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:16.034956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad872d0d-786d-493c-8f29-f521d7fcb2df None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:16.035228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad872d0d-786d-493c-8f29-f521d7fcb2df None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:16.043471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad872d0d-786d-493c-8f29-f521d7fcb2df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:16.044817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad872d0d-786d-493c-8f29-f521d7fcb2df None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:16.050778 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa87dc-b04b-42a0-9ec1-006dfdbab1b4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ad872d0d-786d-493c-8f29-f521d7fcb2df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:16.057231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:16.058697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:16.059625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:16.059625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:16.059625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:16.059625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:16.059625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:16.059625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:16.059625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:16.059625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:16.059625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:16.059625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:16.065613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 280/558] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Aug 6 01:05:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:16.066724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad872d0d-786d-493c-8f29-f521d7fcb2df None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:16.093722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:16.110367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35220795-c7a4-48c9-b5c7-dc6855b57494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:16.111348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35220795-c7a4-48c9-b5c7-dc6855b57494 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:16.111911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35220795-c7a4-48c9-b5c7-dc6855b57494 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:16.112432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35220795-c7a4-48c9-b5c7-dc6855b57494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:16.136202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:16.189661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad872d0d-786d-493c-8f29-f521d7fcb2df None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:16.215912 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35220795-c7a4-48c9-b5c7-dc6855b57494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:16.217881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35220795-c7a4-48c9-b5c7-dc6855b57494 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:16.257426 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fafbac-90a3-4fcc-9c86-253948cbfdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:16.258376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fafbac-90a3-4fcc-9c86-253948cbfdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:16.258487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fafbac-90a3-4fcc-9c86-253948cbfdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:16.258737 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fafbac-90a3-4fcc-9c86-253948cbfdf5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:16.282954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fafbac-90a3-4fcc-9c86-253948cbfdf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-c8fafbac-90a3-4fcc-9c86-253948cbfdf5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:16.290415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:16.291403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:16.292608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 281/559] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:16.293297 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fafbac-90a3-4fcc-9c86-253948cbfdf5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:16.364680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 279/560] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:16.366262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35220795-c7a4-48c9-b5c7-dc6855b57494 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:16.420571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:16.455216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fafbac-90a3-4fcc-9c86-253948cbfdf5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:16.460142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:16.519177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879a2ca-6cd6-4e7e-8366-b852ec360e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:16.520211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879a2ca-6cd6-4e7e-8366-b852ec360e7e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:16.520750 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879a2ca-6cd6-4e7e-8366-b852ec360e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:16.521220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879a2ca-6cd6-4e7e-8366-b852ec360e7e None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:16.522514 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879a2ca-6cd6-4e7e-8366-b852ec360e7e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:16.523464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879a2ca-6cd6-4e7e-8366-b852ec360e7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:16.524941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879a2ca-6cd6-4e7e-8366-b852ec360e7e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:16.559458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 282/561] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Aug 6 01:05:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:16.559882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35220795-c7a4-48c9-b5c7-dc6855b57494 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:16.564800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879a2ca-6cd6-4e7e-8366-b852ec360e7e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:16.606846 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dbed8-e4c7-4af6-8921-d07671a17fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:16.608196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dbed8-e4c7-4af6-8921-d07671a17fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:16.608678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dbed8-e4c7-4af6-8921-d07671a17fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:16.609156 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dbed8-e4c7-4af6-8921-d07671a17fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:16.610067 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60dbed8-e4c7-4af6-8921-d07671a17fb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:16.610572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60dbed8-e4c7-4af6-8921-d07671a17fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:16.612296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60dbed8-e4c7-4af6-8921-d07671a17fb4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:16.622625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 280/562] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:16.623915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dbed8-e4c7-4af6-8921-d07671a17fb4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:16.742336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dbed8-e4c7-4af6-8921-d07671a17fb4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:16.752484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:16.781242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc2514-5b3e-4f13-92f4-929e31c1be72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:16.782266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2514-5b3e-4f13-92f4-929e31c1be72 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:16.782744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2514-5b3e-4f13-92f4-929e31c1be72 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:16.783231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2514-5b3e-4f13-92f4-929e31c1be72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:16.796570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc2514-5b3e-4f13-92f4-929e31c1be72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:16.798456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc2514-5b3e-4f13-92f4-929e31c1be72 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-7acc2514-5b3e-4f13-92f4-929e31c1be72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:16.817387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:16.818296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:16.819186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:16.819186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:16.819186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:16.819186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:16.819459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879a2ca-6cd6-4e7e-8366-b852ec360e7e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:16.826641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 281/563] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:16.826641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc2514-5b3e-4f13-92f4-929e31c1be72 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:16.863169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1375e4b-1ca1-42c5-806b-29d5e6fef5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:16.863908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1375e4b-1ca1-42c5-806b-29d5e6fef5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:16.864200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1375e4b-1ca1-42c5-806b-29d5e6fef5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:16.864711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1375e4b-1ca1-42c5-806b-29d5e6fef5f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:16.867901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:16.876642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1375e4b-1ca1-42c5-806b-29d5e6fef5f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:16.878328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1375e4b-1ca1-42c5-806b-29d5e6fef5f7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e1375e4b-1ca1-42c5-806b-29d5e6fef5f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:16.893804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:16.894763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:16.895649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:16.895649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:16.895649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:16.895649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:16.895649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 283/564] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:16.896428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1375e4b-1ca1-42c5-806b-29d5e6fef5f7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:16.949460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc2514-5b3e-4f13-92f4-929e31c1be72 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:16.986109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d7177-50d5-4c59-967c-e23b806e326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:16.993338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d7177-50d5-4c59-967c-e23b806e326d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:16.993338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d7177-50d5-4c59-967c-e23b806e326d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:16.993338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d7177-50d5-4c59-967c-e23b806e326d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.000073 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d7177-50d5-4c59-967c-e23b806e326d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:17.002259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d7177-50d5-4c59-967c-e23b806e326d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:17.003911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1375e4b-1ca1-42c5-806b-29d5e6fef5f7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f82d7177-50d5-4c59-967c-e23b806e326d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:17.016617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:17.017547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:17.018501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:17.018501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:17.018501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:17.018501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:17.018501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 282/565] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:17.019213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459c005-813a-408f-bc72-7f38629f89eb None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.019458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459c005-813a-408f-bc72-7f38629f89eb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.019728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459c005-813a-408f-bc72-7f38629f89eb None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.020970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 283/566] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:17.021651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459c005-813a-408f-bc72-7f38629f89eb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.038776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:17.047263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4473d-fed7-4938-967e-b5708257f169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.048140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4473d-fed7-4938-967e-b5708257f169 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.048427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4473d-fed7-4938-967e-b5708257f169 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.052581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4473d-fed7-4938-967e-b5708257f169 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.052581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4473d-fed7-4938-967e-b5708257f169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:17.052581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4473d-fed7-4938-967e-b5708257f169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:17.052581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4473d-fed7-4938-967e-b5708257f169 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:17.064276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 284/567] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:17.065483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111de9e5-8198-422b-904c-f13ce8ca1451 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.065744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111de9e5-8198-422b-904c-f13ce8ca1451 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.066020 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111de9e5-8198-422b-904c-f13ce8ca1451 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.071374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 285/568] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:17.072168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111de9e5-8198-422b-904c-f13ce8ca1451 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.122344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:17.125508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459c005-813a-408f-bc72-7f38629f89eb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.158524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756076e-d06f-4e94-b765-8bc007857537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.159046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756076e-d06f-4e94-b765-8bc007857537 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.159326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756076e-d06f-4e94-b765-8bc007857537 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.159600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756076e-d06f-4e94-b765-8bc007857537 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.160320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756076e-d06f-4e94-b765-8bc007857537 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:17.160627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756076e-d06f-4e94-b765-8bc007857537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:17.161992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756076e-d06f-4e94-b765-8bc007857537 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:17.172644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111de9e5-8198-422b-904c-f13ce8ca1451 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.173791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 284/569] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:17.175614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7fd8e-64c1-45e0-bd36-a6dfdb3d6373 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.175614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7fd8e-64c1-45e0-bd36-a6dfdb3d6373 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.175614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7fd8e-64c1-45e0-bd36-a6dfdb3d6373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.192825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-20d7fd8e-64c1-45e0-bd36-a6dfdb3d6373 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:17.194005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-20d7fd8e-64c1-45e0-bd36-a6dfdb3d6373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:17.207558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf71f87-0c20-4bf6-bf82-b9e64e31316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.208208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71f87-0c20-4bf6-bf82-b9e64e31316b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.208499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71f87-0c20-4bf6-bf82-b9e64e31316b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.208775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf71f87-0c20-4bf6-bf82-b9e64e31316b None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.209465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf71f87-0c20-4bf6-bf82-b9e64e31316b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:17.209777 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf71f87-0c20-4bf6-bf82-b9e64e31316b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:17.211689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf71f87-0c20-4bf6-bf82-b9e64e31316b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:17.222671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 286/570] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:17.223446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf71f87-0c20-4bf6-bf82-b9e64e31316b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.263359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 285/571] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:17.265005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af6b58-d940-481b-8b70-c1d750ea8a9c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.265296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af6b58-d940-481b-8b70-c1d750ea8a9c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.265575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af6b58-d940-481b-8b70-c1d750ea8a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.280120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:17.282765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-18af6b58-d940-481b-8b70-c1d750ea8a9c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:17.283967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-18af6b58-d940-481b-8b70-c1d750ea8a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:17.320163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf71f87-0c20-4bf6-bf82-b9e64e31316b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.329001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:17.350351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 286/572] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:17.352064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af6b58-d940-481b-8b70-c1d750ea8a9c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.365495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcbe7df-d71b-48de-b662-cce7efa77c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.366748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbe7df-d71b-48de-b662-cce7efa77c35 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.367036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbe7df-d71b-48de-b662-cce7efa77c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.367314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbe7df-d71b-48de-b662-cce7efa77c35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.374126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcbe7df-d71b-48de-b662-cce7efa77c35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:17.375578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcbe7df-d71b-48de-b662-cce7efa77c35 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-cbcbe7df-d71b-48de-b662-cce7efa77c35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:17.386611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:17.387667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:17.388897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:17.388897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:17.388897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:17.388897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:17.388897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:17.388897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:17.388897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:17.388897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:17.388897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 287/573] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:17.388897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcbe7df-d71b-48de-b662-cce7efa77c35 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.452193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af6b58-d940-481b-8b70-c1d750ea8a9c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.482710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:17.486733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcbe7df-d71b-48de-b662-cce7efa77c35 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.487256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff0b2d-eb12-42f9-ba5b-6b7d2079be66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.487925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff0b2d-eb12-42f9-ba5b-6b7d2079be66 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.488210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff0b2d-eb12-42f9-ba5b-6b7d2079be66 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.488481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff0b2d-eb12-42f9-ba5b-6b7d2079be66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.495191 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff0b2d-eb12-42f9-ba5b-6b7d2079be66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:17.496665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff0b2d-eb12-42f9-ba5b-6b7d2079be66 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-94ff0b2d-eb12-42f9-ba5b-6b7d2079be66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:17.507823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:17.508734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:17.509623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:17.509623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:17.509623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:17.509623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:17.509623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:17.509623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:17.509623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:17.509623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:17.509623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 287/574] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:17.510355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff0b2d-eb12-42f9-ba5b-6b7d2079be66 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.521880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1bea1-9e4c-4bf1-9d7e-ace52bfb631c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.523082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1bea1-9e4c-4bf1-9d7e-ace52bfb631c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.523370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1bea1-9e4c-4bf1-9d7e-ace52bfb631c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.523647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1bea1-9e4c-4bf1-9d7e-ace52bfb631c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.531022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1bea1-9e4c-4bf1-9d7e-ace52bfb631c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:17.532498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1bea1-9e4c-4bf1-9d7e-ace52bfb631c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-41d1bea1-9e4c-4bf1-9d7e-ace52bfb631c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:17.543857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:17.544943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 288/575] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:17.545913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1bea1-9e4c-4bf1-9d7e-ace52bfb631c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.579455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:17.609654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff0b2d-eb12-42f9-ba5b-6b7d2079be66 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.643889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e716e2-991d-4fa7-8d29-db25aae8dabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.644530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e716e2-991d-4fa7-8d29-db25aae8dabe None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.644814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e716e2-991d-4fa7-8d29-db25aae8dabe None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.645089 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e716e2-991d-4fa7-8d29-db25aae8dabe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.652228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1bea1-9e4c-4bf1-9d7e-ace52bfb631c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.655515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e716e2-991d-4fa7-8d29-db25aae8dabe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-84e716e2-991d-4fa7-8d29-db25aae8dabe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:17.661787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:17.662781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:17.663634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 288/576] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:17.664386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e716e2-991d-4fa7-8d29-db25aae8dabe None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.690428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395f7fb-b869-43aa-815c-1bb5401ad76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.691074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395f7fb-b869-43aa-815c-1bb5401ad76b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.691364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395f7fb-b869-43aa-815c-1bb5401ad76b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.691634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395f7fb-b869-43aa-815c-1bb5401ad76b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.702383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395f7fb-b869-43aa-815c-1bb5401ad76b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-3395f7fb-b869-43aa-815c-1bb5401ad76b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:17.707213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:17.708148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:17.709026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 289/577] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:17.709724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395f7fb-b869-43aa-815c-1bb5401ad76b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.720604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:17.766206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e716e2-991d-4fa7-8d29-db25aae8dabe None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.799698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ba40c-1d10-4100-88e2-218038352d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.799878 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:17.800338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ba40c-1d10-4100-88e2-218038352d94 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.800622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ba40c-1d10-4100-88e2-218038352d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.800901 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ba40c-1d10-4100-88e2-218038352d94 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.801646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ba40c-1d10-4100-88e2-218038352d94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:17.801957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ba40c-1d10-4100-88e2-218038352d94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:17.803381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ba40c-1d10-4100-88e2-218038352d94 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:17.811116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395f7fb-b869-43aa-815c-1bb5401ad76b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.815027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 289/578] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:17.815840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ba40c-1d10-4100-88e2-218038352d94 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.845173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda316f-1cfb-42b7-a3a5-4c861dcb0ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.845798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda316f-1cfb-42b7-a3a5-4c861dcb0ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.846082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda316f-1cfb-42b7-a3a5-4c861dcb0ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.846357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda316f-1cfb-42b7-a3a5-4c861dcb0ba1 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.847103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda316f-1cfb-42b7-a3a5-4c861dcb0ba1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:17.847414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda316f-1cfb-42b7-a3a5-4c861dcb0ba1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:17.848892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda316f-1cfb-42b7-a3a5-4c861dcb0ba1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:17.860183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 290/579] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:17.860957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda316f-1cfb-42b7-a3a5-4c861dcb0ba1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.921937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ba40c-1d10-4100-88e2-218038352d94 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:17.937276 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:17.958123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e21e0-d6b4-43b8-b6f8-ac15cb7f6d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:17.958677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e21e0-d6b4-43b8-b6f8-ac15cb7f6d4e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:17.958956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e21e0-d6b4-43b8-b6f8-ac15cb7f6d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:17.959232 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e21e0-d6b4-43b8-b6f8-ac15cb7f6d4e None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:17.960043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e21e0-d6b4-43b8-b6f8-ac15cb7f6d4e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:17.960353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e21e0-d6b4-43b8-b6f8-ac15cb7f6d4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:17.961763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e21e0-d6b4-43b8-b6f8-ac15cb7f6d4e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:17.978360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 290/580] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:17.979484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e21e0-d6b4-43b8-b6f8-ac15cb7f6d4e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:17.992205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda316f-1cfb-42b7-a3a5-4c861dcb0ba1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.024559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb458-b5c0-4bec-b21c-7a478b1600a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:18.025175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb458-b5c0-4bec-b21c-7a478b1600a4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:18.025456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb458-b5c0-4bec-b21c-7a478b1600a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:18.025730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb458-b5c0-4bec-b21c-7a478b1600a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:18.026446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fb458-b5c0-4bec-b21c-7a478b1600a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:18.026753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fb458-b5c0-4bec-b21c-7a478b1600a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:18.028170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fb458-b5c0-4bec-b21c-7a478b1600a4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:18.036143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 291/581] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:18.036898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb458-b5c0-4bec-b21c-7a478b1600a4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:18.048661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:18.076127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e21e0-d6b4-43b8-b6f8-ac15cb7f6d4e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.110618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d9e72-58aa-400a-ba4f-6a433c3c79ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:18.111237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d9e72-58aa-400a-ba4f-6a433c3c79ce None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:18.111517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d9e72-58aa-400a-ba4f-6a433c3c79ce None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:18.111811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d9e72-58aa-400a-ba4f-6a433c3c79ce None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:18.112522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d9e72-58aa-400a-ba4f-6a433c3c79ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:18.112825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d9e72-58aa-400a-ba4f-6a433c3c79ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:18.114172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d9e72-58aa-400a-ba4f-6a433c3c79ce None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:18.122868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 291/582] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:18.123644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d9e72-58aa-400a-ba4f-6a433c3c79ce None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:18.141555 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb458-b5c0-4bec-b21c-7a478b1600a4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.154840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:18.174064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e48ba0-664a-45f8-90e6-196c4c2f777f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:18.174727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e48ba0-664a-45f8-90e6-196c4c2f777f None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:18.181705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e48ba0-664a-45f8-90e6-196c4c2f777f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:18.181705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e48ba0-664a-45f8-90e6-196c4c2f777f None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/db1b205d165543d0a3a69a2db6af5d1c/roles/4dabc11cde5340ee9eb2c28e6154b3ec` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:18.189782 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e48ba0-664a-45f8-90e6-196c4c2f777f None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=db1b205d165543d0a3a69a2db6af5d1c, role_id=4dabc11cde5340ee9eb2c28e6154b3ec)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:18.191232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e48ba0-664a-45f8-90e6-196c4c2f777f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:18.221099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d9e72-58aa-400a-ba4f-6a433c3c79ce None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.253904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a32c43-d4b6-4f67-ac7a-569a77dd16f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:18.254511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32c43-d4b6-4f67-ac7a-569a77dd16f1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:18.254795 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32c43-d4b6-4f67-ac7a-569a77dd16f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:18.255072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32c43-d4b6-4f67-ac7a-569a77dd16f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:18.255888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a32c43-d4b6-4f67-ac7a-569a77dd16f1 None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=33aa626411f1474595102a4541af901a) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:18.256204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a32c43-d4b6-4f67-ac7a-569a77dd16f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:18.258278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a32c43-d4b6-4f67-ac7a-569a77dd16f1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:18.263537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 292/583] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:18 2022] GET /identity/v3/role_assignments?user.id=33aa626411f1474595102a4541af901a&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:18.264204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 292/584] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:05:18 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/db1b205d165543d0a3a69a2db6af5d1c/roles/4dabc11cde5340ee9eb2c28e6154b3ec => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:05:18.264435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a32c43-d4b6-4f67-ac7a-569a77dd16f1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:18.264809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e48ba0-664a-45f8-90e6-196c4c2f777f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:18.348288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:18.387328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e48ba0-664a-45f8-90e6-196c4c2f777f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.398467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:18.406020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a32c43-d4b6-4f67-ac7a-569a77dd16f1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.428314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e747c-cddf-41aa-a8ea-ba8960742e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:18.429548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e747c-cddf-41aa-a8ea-ba8960742e63 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:18.429833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e747c-cddf-41aa-a8ea-ba8960742e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:18.430112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e747c-cddf-41aa-a8ea-ba8960742e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:18.443161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e747c-cddf-41aa-a8ea-ba8960742e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:18.445212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e747c-cddf-41aa-a8ea-ba8960742e63 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:18.456575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871dad5-19eb-4c85-a16b-b8098948f1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:18.457313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871dad5-19eb-4c85-a16b-b8098948f1bd None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:18.457595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871dad5-19eb-4c85-a16b-b8098948f1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:18.457953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871dad5-19eb-4c85-a16b-b8098948f1bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-553e747c-cddf-41aa-a8ea-ba8960742e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:18.463509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:18.464427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:18.465327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:18.465327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:18.465327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:18.465327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:18.465327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 293/585] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:18.468883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e747c-cddf-41aa-a8ea-ba8960742e63 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:18.470287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871dad5-19eb-4c85-a16b-b8098948f1bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:18.472343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871dad5-19eb-4c85-a16b-b8098948f1bd None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0871dad5-19eb-4c85-a16b-b8098948f1bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:18.490469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:18.492260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:18.493371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:18.493371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:18.493371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:18.493371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:18.494802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 293/586] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:18.495736 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871dad5-19eb-4c85-a16b-b8098948f1bd None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:18.603953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871dad5-19eb-4c85-a16b-b8098948f1bd None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.621948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e747c-cddf-41aa-a8ea-ba8960742e63 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.638608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ceea3-1fd4-4d57-b0b7-dc89b1270781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:18.639067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:18.639621 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ceea3-1fd4-4d57-b0b7-dc89b1270781 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:18.639941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ceea3-1fd4-4d57-b0b7-dc89b1270781 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:18.640314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ceea3-1fd4-4d57-b0b7-dc89b1270781 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:18.641183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ceea3-1fd4-4d57-b0b7-dc89b1270781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:18.641549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ceea3-1fd4-4d57-b0b7-dc89b1270781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:18.642996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ceea3-1fd4-4d57-b0b7-dc89b1270781 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:18.655998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 294/587] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:18.657026 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ceea3-1fd4-4d57-b0b7-dc89b1270781 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:18.665272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f59b7-ce32-43ae-9495-369787ef6cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:18.666138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f59b7-ce32-43ae-9495-369787ef6cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:18.667076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f59b7-ce32-43ae-9495-369787ef6cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:18.667519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f59b7-ce32-43ae-9495-369787ef6cf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:18.669493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:18.684179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f59b7-ce32-43ae-9495-369787ef6cf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:18.690350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f59b7-ce32-43ae-9495-369787ef6cf6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-9a1f59b7-ce32-43ae-9495-369787ef6cf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:18.705425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:18.706362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:18.707267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:18.707267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:18.707267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:18.707267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:18.707267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 294/588] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:18.707558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f59b7-ce32-43ae-9495-369787ef6cf6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:18.787151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ceea3-1fd4-4d57-b0b7-dc89b1270781 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.824604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8d84d-ac89-4bbf-b385-4de9a004d381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:18.825468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8d84d-ac89-4bbf-b385-4de9a004d381 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:18.825936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8d84d-ac89-4bbf-b385-4de9a004d381 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:18.826413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8d84d-ac89-4bbf-b385-4de9a004d381 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:18.827555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8d84d-ac89-4bbf-b385-4de9a004d381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:18.828073 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8d84d-ac89-4bbf-b385-4de9a004d381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:18.829616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8d84d-ac89-4bbf-b385-4de9a004d381 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:18.842046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 295/589] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:18.843241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8d84d-ac89-4bbf-b385-4de9a004d381 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:18.847994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f59b7-ce32-43ae-9495-369787ef6cf6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.895518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c6864-0c4a-481d-8a3b-333d826125a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:18.896199 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c6864-0c4a-481d-8a3b-333d826125a6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:18.896715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c6864-0c4a-481d-8a3b-333d826125a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:18.897004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c6864-0c4a-481d-8a3b-333d826125a6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:18.907661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c6864-0c4a-481d-8a3b-333d826125a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:18.909858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c6864-0c4a-481d-8a3b-333d826125a6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:18.917124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b82c6864-0c4a-481d-8a3b-333d826125a6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:18.925373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:18.926435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:18.927377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 295/590] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:18.930627 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c6864-0c4a-481d-8a3b-333d826125a6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:18.963107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8d84d-ac89-4bbf-b385-4de9a004d381 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:18.963500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:18.999379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f72df1-9fb1-4b52-a311-4786876f1d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:19.000418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f72df1-9fb1-4b52-a311-4786876f1d90 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:19.000913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f72df1-9fb1-4b52-a311-4786876f1d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:19.001389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f72df1-9fb1-4b52-a311-4786876f1d90 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:19.002299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f72df1-9fb1-4b52-a311-4786876f1d90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:19.002806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f72df1-9fb1-4b52-a311-4786876f1d90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:19.004430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f72df1-9fb1-4b52-a311-4786876f1d90 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:19.021694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 296/591] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:19.023459 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f72df1-9fb1-4b52-a311-4786876f1d90 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:19.044518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c6864-0c4a-481d-8a3b-333d826125a6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:19.081362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad51af9-3d2c-4186-844d-d6012f80dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:19.082009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad51af9-3d2c-4186-844d-d6012f80dc9b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:19.082292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad51af9-3d2c-4186-844d-d6012f80dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:19.082572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad51af9-3d2c-4186-844d-d6012f80dc9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:19.090469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad51af9-3d2c-4186-844d-d6012f80dc9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:19.092075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad51af9-3d2c-4186-844d-d6012f80dc9b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-fad51af9-3d2c-4186-844d-d6012f80dc9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:19.103890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:19.104773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:19.105667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:19.105667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:19.105667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:19.105667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:19.105667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:19.105667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:19.105667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:19.105667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:19.105667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 296/592] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:19.106586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad51af9-3d2c-4186-844d-d6012f80dc9b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:19.135245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f72df1-9fb1-4b52-a311-4786876f1d90 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:19.163241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:19.171913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe079f4-a20e-4265-8e0b-c13766c2650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:19.173019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe079f4-a20e-4265-8e0b-c13766c2650f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:19.173148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe079f4-a20e-4265-8e0b-c13766c2650f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:19.173471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe079f4-a20e-4265-8e0b-c13766c2650f None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:19.174345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe079f4-a20e-4265-8e0b-c13766c2650f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:19.174700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe079f4-a20e-4265-8e0b-c13766c2650f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:19.176227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe079f4-a20e-4265-8e0b-c13766c2650f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:19.193942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 297/593] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Aug 6 01:05:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:19.195613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe079f4-a20e-4265-8e0b-c13766c2650f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:19.202205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:19.220595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad51af9-3d2c-4186-844d-d6012f80dc9b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:19.256239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c122b1-07e7-4094-ab02-e2d16ac60a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:19.257737 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c122b1-07e7-4094-ab02-e2d16ac60a73 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:19.258162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c122b1-07e7-4094-ab02-e2d16ac60a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:19.258573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c122b1-07e7-4094-ab02-e2d16ac60a73 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:19.266275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c122b1-07e7-4094-ab02-e2d16ac60a73 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:19.268095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c122b1-07e7-4094-ab02-e2d16ac60a73 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-09c122b1-07e7-4094-ab02-e2d16ac60a73 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:19.280722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:19.281657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:19.282609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:19.282609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:19.282609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:19.282609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:19.282609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:19.282609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:19.282609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:19.282609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:19.282609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:05:19.282609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:19.285347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 297/594] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:19.286313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c122b1-07e7-4094-ab02-e2d16ac60a73 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:19.304854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe079f4-a20e-4265-8e0b-c13766c2650f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:19.342347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e414fce-718e-489d-a9ba-97daa4370931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:19.343242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e414fce-718e-489d-a9ba-97daa4370931 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:19.343696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e414fce-718e-489d-a9ba-97daa4370931 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:19.344159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e414fce-718e-489d-a9ba-97daa4370931 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:19.345109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e414fce-718e-489d-a9ba-97daa4370931 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:19.345591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e414fce-718e-489d-a9ba-97daa4370931 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:19.347182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e414fce-718e-489d-a9ba-97daa4370931 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:19.364431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 298/595] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:19.364431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e414fce-718e-489d-a9ba-97daa4370931 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:19.404383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c122b1-07e7-4094-ab02-e2d16ac60a73 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:19.414745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:19.459899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cc2db-ae02-453b-90d6-44ecd5576714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:19.460857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cc2db-ae02-453b-90d6-44ecd5576714 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:19.461284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cc2db-ae02-453b-90d6-44ecd5576714 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:19.461697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cc2db-ae02-453b-90d6-44ecd5576714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:19.474614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e414fce-718e-489d-a9ba-97daa4370931 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:19.477713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cc2db-ae02-453b-90d6-44ecd5576714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:19.479328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cc2db-ae02-453b-90d6-44ecd5576714 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:19.483834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-531cc2db-ae02-453b-90d6-44ecd5576714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:19.501458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:19.502549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:19.503634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:19.503634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:19.503634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:19.503634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:19.510260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 298/596] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:19.510260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cc2db-ae02-453b-90d6-44ecd5576714 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:19.521811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ecb053-fe51-40b6-8206-697f3ecd4cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:19.522612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecb053-fe51-40b6-8206-697f3ecd4cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:19.522995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecb053-fe51-40b6-8206-697f3ecd4cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:19.523375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecb053-fe51-40b6-8206-697f3ecd4cb1 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:19.524423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ecb053-fe51-40b6-8206-697f3ecd4cb1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:19.524855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ecb053-fe51-40b6-8206-697f3ecd4cb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:19.526509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ecb053-fe51-40b6-8206-697f3ecd4cb1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:19.545524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 299/597] 10.222.0.25 () {56 vars in 1139 bytes} [Sat Aug 6 01:05:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:19.546561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ecb053-fe51-40b6-8206-697f3ecd4cb1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:19.626231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cc2db-ae02-453b-90d6-44ecd5576714 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:19.654406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ecb053-fe51-40b6-8206-697f3ecd4cb1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:19.657150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:19.662093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0aa013-7992-4620-b39a-52698058de8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:19.662924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0aa013-7992-4620-b39a-52698058de8c None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:19.663382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0aa013-7992-4620-b39a-52698058de8c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:19.663818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0aa013-7992-4620-b39a-52698058de8c None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/ddd0e588d52e40f08a2dc1062ecd4232/roles/4dabc11cde5340ee9eb2c28e6154b3ec` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:19.678271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0aa013-7992-4620-b39a-52698058de8c None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=ddd0e588d52e40f08a2dc1062ecd4232, role_id=4dabc11cde5340ee9eb2c28e6154b3ec)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:19.680597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0aa013-7992-4620-b39a-52698058de8c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:19.692045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef297ac-16a5-4309-867c-c0394581922c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:19.692744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef297ac-16a5-4309-867c-c0394581922c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:19.693270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef297ac-16a5-4309-867c-c0394581922c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:19.693535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef297ac-16a5-4309-867c-c0394581922c None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:19.694338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef297ac-16a5-4309-867c-c0394581922c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:19.695225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef297ac-16a5-4309-867c-c0394581922c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:19.696669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef297ac-16a5-4309-867c-c0394581922c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:19.713290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 300/598] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:19.713290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef297ac-16a5-4309-867c-c0394581922c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:19.744047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 299/599] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:05:19 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/ddd0e588d52e40f08a2dc1062ecd4232/roles/4dabc11cde5340ee9eb2c28e6154b3ec => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:05:19.745575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0aa013-7992-4620-b39a-52698058de8c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:19.830489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef297ac-16a5-4309-867c-c0394581922c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:19.836080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:19.877225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ab3f5-e05a-4de9-bad8-c08010daa3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:19.877891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab3f5-e05a-4de9-bad8-c08010daa3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:19.878177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab3f5-e05a-4de9-bad8-c08010daa3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:19.878459 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab3f5-e05a-4de9-bad8-c08010daa3c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:19.906439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ab3f5-e05a-4de9-bad8-c08010daa3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:19.907972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ab3f5-e05a-4de9-bad8-c08010daa3c6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3b5ab3f5-e05a-4de9-bad8-c08010daa3c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:19.942239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:19.943198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:19.944813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:19.944813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:19.944813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:19.944813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:19.944813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 301/600] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:19.951749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ab3f5-e05a-4de9-bad8-c08010daa3c6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:19.971857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0aa013-7992-4620-b39a-52698058de8c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:19.991032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:20.019052 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9454f0-7c51-44dc-b22a-f234a0023bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:20.019711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9454f0-7c51-44dc-b22a-f234a0023bab None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.020041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9454f0-7c51-44dc-b22a-f234a0023bab None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.020481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9454f0-7c51-44dc-b22a-f234a0023bab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.034510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9454f0-7c51-44dc-b22a-f234a0023bab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:20.036387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9454f0-7c51-44dc-b22a-f234a0023bab None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-bd9454f0-7c51-44dc-b22a-f234a0023bab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:20.049993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:20.050931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:20.051838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:20.051838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:20.051838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:20.051838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:20.051838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:20.051838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:20.051838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:20.051838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:20.061009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 300/601] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:20.062568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9454f0-7c51-44dc-b22a-f234a0023bab None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:20.094776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ab3f5-e05a-4de9-bad8-c08010daa3c6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:20.144515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b828a-f679-44eb-a948-0f289f006f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:20.146462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b828a-f679-44eb-a948-0f289f006f9d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.147101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b828a-f679-44eb-a948-0f289f006f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.148815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b828a-f679-44eb-a948-0f289f006f9d None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.149749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b828a-f679-44eb-a948-0f289f006f9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:20.150927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b828a-f679-44eb-a948-0f289f006f9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:20.153220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b828a-f679-44eb-a948-0f289f006f9d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:20.176140 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 302/602] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:20.182422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b828a-f679-44eb-a948-0f289f006f9d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:20.238242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9454f0-7c51-44dc-b22a-f234a0023bab None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:20.249202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:20.290372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae774b9-dfd6-4256-aba2-f3eafc404e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:20.291394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae774b9-dfd6-4256-aba2-f3eafc404e05 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.291934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae774b9-dfd6-4256-aba2-f3eafc404e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.292403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae774b9-dfd6-4256-aba2-f3eafc404e05 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.293492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae774b9-dfd6-4256-aba2-f3eafc404e05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:20.294380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae774b9-dfd6-4256-aba2-f3eafc404e05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:20.297342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae774b9-dfd6-4256-aba2-f3eafc404e05 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:20.315920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 301/603] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:20.317978 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae774b9-dfd6-4256-aba2-f3eafc404e05 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:20.362012 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:20.401024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b828a-f679-44eb-a948-0f289f006f9d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:20.427166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae774b9-dfd6-4256-aba2-f3eafc404e05 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:20.438055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf874c9e-ed15-4d92-8791-a1708f8e1217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:20.438706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf874c9e-ed15-4d92-8791-a1708f8e1217 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.438998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf874c9e-ed15-4d92-8791-a1708f8e1217 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.439554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf874c9e-ed15-4d92-8791-a1708f8e1217 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.446397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf874c9e-ed15-4d92-8791-a1708f8e1217 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:20.448203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf874c9e-ed15-4d92-8791-a1708f8e1217 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bf874c9e-ed15-4d92-8791-a1708f8e1217 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:20.460953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:20.461861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:20.462763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:20.462763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:20.462763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:20.462763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:20.462763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:20.462763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:20.462763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:20.462763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:20.462763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 303/604] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:20.463838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d58f7-70ba-44ce-9f2e-0e5bc08689c1 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.463838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d58f7-70ba-44ce-9f2e-0e5bc08689c1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.463838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d58f7-70ba-44ce-9f2e-0e5bc08689c1 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.467947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 304/605] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:20.467947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b801f-5fbe-416b-bffd-2fb1657d7784 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.467947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b801f-5fbe-416b-bffd-2fb1657d7784 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.467947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b801f-5fbe-416b-bffd-2fb1657d7784 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.467947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 305/606] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:20.469136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b801f-5fbe-416b-bffd-2fb1657d7784 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:20.474012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6b533-8607-49d9-b087-045a736478e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:20.474705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6b533-8607-49d9-b087-045a736478e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.474993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6b533-8607-49d9-b087-045a736478e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.475273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6b533-8607-49d9-b087-045a736478e7 None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/9a234a96eccd4f38b0213d7d0461ad24/roles/4dabc11cde5340ee9eb2c28e6154b3ec` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.491869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6b533-8607-49d9-b087-045a736478e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=9a234a96eccd4f38b0213d7d0461ad24, role_id=4dabc11cde5340ee9eb2c28e6154b3ec)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:20.493346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6b533-8607-49d9-b087-045a736478e7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:20.572193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b801f-5fbe-416b-bffd-2fb1657d7784 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:20.590572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 302/607] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:05:20 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/9a234a96eccd4f38b0213d7d0461ad24/roles/4dabc11cde5340ee9eb2c28e6154b3ec => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:05:20.592074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f5c4f-e0c1-4afe-915c-b253cb28c3a3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.592512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f5c4f-e0c1-4afe-915c-b253cb28c3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.592802 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f5c4f-e0c1-4afe-915c-b253cb28c3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.618243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154db0c-2e63-42dd-88aa-9362a02245c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:20.619208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154db0c-2e63-42dd-88aa-9362a02245c5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.619690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154db0c-2e63-42dd-88aa-9362a02245c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.619933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-683f5c4f-e0c1-4afe-915c-b253cb28c3a3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:20.620383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154db0c-2e63-42dd-88aa-9362a02245c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.621066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-683f5c4f-e0c1-4afe-915c-b253cb28c3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:20.621585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154db0c-2e63-42dd-88aa-9362a02245c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:20.622149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154db0c-2e63-42dd-88aa-9362a02245c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:20.624083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154db0c-2e63-42dd-88aa-9362a02245c5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:20.633718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:20.636608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:20.640205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 306/608] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:20.645343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6cf4f-3d77-4efa-892e-f4e2bf7b8d96 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.645699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6cf4f-3d77-4efa-892e-f4e2bf7b8d96 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.645908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6cf4f-3d77-4efa-892e-f4e2bf7b8d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.667753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c4d6cf4f-3d77-4efa-892e-f4e2bf7b8d96 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:20.669077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c4d6cf4f-3d77-4efa-892e-f4e2bf7b8d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:20.717296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 303/609] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:20.718579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f5c4f-e0c1-4afe-915c-b253cb28c3a3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:20.747325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 307/610] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:20.760359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6cf4f-3d77-4efa-892e-f4e2bf7b8d96 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:20.852855 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f5c4f-e0c1-4afe-915c-b253cb28c3a3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:20.888913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62311538-aef1-462e-8fdd-a35024b10ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:20.889514 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62311538-aef1-462e-8fdd-a35024b10ad4 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.889792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62311538-aef1-462e-8fdd-a35024b10ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.890069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62311538-aef1-462e-8fdd-a35024b10ad4 None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/2c14fb133a0f419390fb0d00c600bdd4/roles/4dabc11cde5340ee9eb2c28e6154b3ec` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.892945 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6cf4f-3d77-4efa-892e-f4e2bf7b8d96 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:20.905995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62311538-aef1-462e-8fdd-a35024b10ad4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=2c14fb133a0f419390fb0d00c600bdd4, role_id=4dabc11cde5340ee9eb2c28e6154b3ec)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:20.907596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62311538-aef1-462e-8fdd-a35024b10ad4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:20.911176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:20.926869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:20.936412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6147c-7eda-40f8-a348-3b248a732b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:20.937057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6147c-7eda-40f8-a348-3b248a732b08 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:20.937339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6147c-7eda-40f8-a348-3b248a732b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:20.937616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6147c-7eda-40f8-a348-3b248a732b08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:20.948151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6147c-7eda-40f8-a348-3b248a732b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-11a6147c-7eda-40f8-a348-3b248a732b08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:20.953244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:20.954210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:20.955064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 308/611] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:20.955815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6147c-7eda-40f8-a348-3b248a732b08 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:20.990081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 304/612] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:05:20 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/2c14fb133a0f419390fb0d00c600bdd4/roles/4dabc11cde5340ee9eb2c28e6154b3ec => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:05:20.992004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62311538-aef1-462e-8fdd-a35024b10ad4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:21.086120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6147c-7eda-40f8-a348-3b248a732b08 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:21.127934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62311538-aef1-462e-8fdd-a35024b10ad4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:21.160463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515e0c1-d9c7-4899-a933-b5537b696065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:21.161178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515e0c1-d9c7-4899-a933-b5537b696065 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:21.161563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515e0c1-d9c7-4899-a933-b5537b696065 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:21.161843 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515e0c1-d9c7-4899-a933-b5537b696065 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:21.170210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2f2dd-cd01-4d18-8cc9-4e2ba7b2dcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:21.171491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2f2dd-cd01-4d18-8cc9-4e2ba7b2dcff None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:21.171905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2f2dd-cd01-4d18-8cc9-4e2ba7b2dcff None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:21.172285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2f2dd-cd01-4d18-8cc9-4e2ba7b2dcff None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:21.173281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2f2dd-cd01-4d18-8cc9-4e2ba7b2dcff None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:21.173698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2f2dd-cd01-4d18-8cc9-4e2ba7b2dcff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:21.175371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2f2dd-cd01-4d18-8cc9-4e2ba7b2dcff None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:21.183273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515e0c1-d9c7-4899-a933-b5537b696065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:21.186714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 305/613] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b515e0c1-d9c7-4899-a933-b5537b696065 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:21.192879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:21.194050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:21.195720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 309/614] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:21.196606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2f2dd-cd01-4d18-8cc9-4e2ba7b2dcff None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:21.203223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515e0c1-d9c7-4899-a933-b5537b696065 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:21.288623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:21.294299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:21.356307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2f2dd-cd01-4d18-8cc9-4e2ba7b2dcff None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:21.368719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515e0c1-d9c7-4899-a933-b5537b696065 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:21.403555 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08bc76-5817-425c-a882-d2b58749f44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:21.405308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08bc76-5817-425c-a882-d2b58749f44a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:21.407470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08bc76-5817-425c-a882-d2b58749f44a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:21.407941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08bc76-5817-425c-a882-d2b58749f44a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:21.428475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08bc76-5817-425c-a882-d2b58749f44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:21.430872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08bc76-5817-425c-a882-d2b58749f44a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:21.439502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ae556-29a0-4776-a8f6-857fe788cfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:21.442287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ae556-29a0-4776-a8f6-857fe788cfdf None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:21.442755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ae556-29a0-4776-a8f6-857fe788cfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:21.443193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ae556-29a0-4776-a8f6-857fe788cfdf None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:21.445871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ae556-29a0-4776-a8f6-857fe788cfdf None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:21.446334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ae556-29a0-4776-a8f6-857fe788cfdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:21.448168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ae556-29a0-4776-a8f6-857fe788cfdf None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-cf08bc76-5817-425c-a882-d2b58749f44a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:21.463498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:21.464643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:21.465749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:21.465749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:21.465749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:21.465749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:21.468947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 306/615] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:21.469640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 310/616] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:21.475023 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ae556-29a0-4776-a8f6-857fe788cfdf None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:21.477191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08bc76-5817-425c-a882-d2b58749f44a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:21.648481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08bc76-5817-425c-a882-d2b58749f44a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:21.666314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:21.693384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ae556-29a0-4776-a8f6-857fe788cfdf None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:21.715919 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04d907-6afd-4a86-bd29-2fa5265a64d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:21.716929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04d907-6afd-4a86-bd29-2fa5265a64d5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:21.718650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04d907-6afd-4a86-bd29-2fa5265a64d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:21.719371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04d907-6afd-4a86-bd29-2fa5265a64d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:21.721296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04d907-6afd-4a86-bd29-2fa5265a64d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:21.721814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04d907-6afd-4a86-bd29-2fa5265a64d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:21.723998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04d907-6afd-4a86-bd29-2fa5265a64d5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:21.724111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:21.743591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 307/617] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:21.745206 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04d907-6afd-4a86-bd29-2fa5265a64d5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:21.797925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1fbd0-7164-42fe-9f84-7138a4eb7f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:21.798594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1fbd0-7164-42fe-9f84-7138a4eb7f39 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:21.798845 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1fbd0-7164-42fe-9f84-7138a4eb7f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:21.799122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1fbd0-7164-42fe-9f84-7138a4eb7f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:21.820016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1fbd0-7164-42fe-9f84-7138a4eb7f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:21.821118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1fbd0-7164-42fe-9f84-7138a4eb7f39 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8ba1fbd0-7164-42fe-9f84-7138a4eb7f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:21.847200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:21.848426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:21.849338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:21.849338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:21.849338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:21.849338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:21.851503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 311/618] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:21.854473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1fbd0-7164-42fe-9f84-7138a4eb7f39 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:21.862283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04d907-6afd-4a86-bd29-2fa5265a64d5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:21.903098 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439b2cf-7aaa-4ee2-8551-9512e178b840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:21.904131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439b2cf-7aaa-4ee2-8551-9512e178b840 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:21.905477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439b2cf-7aaa-4ee2-8551-9512e178b840 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:21.905477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439b2cf-7aaa-4ee2-8551-9512e178b840 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:21.925155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439b2cf-7aaa-4ee2-8551-9512e178b840 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:21.926916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439b2cf-7aaa-4ee2-8551-9512e178b840 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d439b2cf-7aaa-4ee2-8551-9512e178b840 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:21.944215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:21.945252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:21.946225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 308/619] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:21.951458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439b2cf-7aaa-4ee2-8551-9512e178b840 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:21.995503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1fbd0-7164-42fe-9f84-7138a4eb7f39 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:22.013424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:22.041737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9d6d5-d568-4da0-9350-8b1daf60af13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:22.042362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9d6d5-d568-4da0-9350-8b1daf60af13 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.042639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9d6d5-d568-4da0-9350-8b1daf60af13 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.042907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9d6d5-d568-4da0-9350-8b1daf60af13 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.043615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9d6d5-d568-4da0-9350-8b1daf60af13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:22.043947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9d6d5-d568-4da0-9350-8b1daf60af13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:22.045318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9d6d5-d568-4da0-9350-8b1daf60af13 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:22.074246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 312/620] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:22.075523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbe070-2e4e-42da-bbaf-112ef80ff417 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.075825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbe070-2e4e-42da-bbaf-112ef80ff417 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.076107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbe070-2e4e-42da-bbaf-112ef80ff417 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.077573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 313/621] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:22.084726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbe070-2e4e-42da-bbaf-112ef80ff417 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:22.110377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:22.117177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439b2cf-7aaa-4ee2-8551-9512e178b840 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:22.160876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d20993-4ec1-42e0-aa6c-f5ead0a15761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:22.161594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d20993-4ec1-42e0-aa6c-f5ead0a15761 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.162018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d20993-4ec1-42e0-aa6c-f5ead0a15761 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.162395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d20993-4ec1-42e0-aa6c-f5ead0a15761 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.163335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d20993-4ec1-42e0-aa6c-f5ead0a15761 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:22.163717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d20993-4ec1-42e0-aa6c-f5ead0a15761 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:22.165479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d20993-4ec1-42e0-aa6c-f5ead0a15761 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:22.185198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 309/622] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Aug 6 01:05:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:22.187048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab5f22-ca0d-4086-b335-1a14e76afcbf None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.187336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab5f22-ca0d-4086-b335-1a14e76afcbf None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.187611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab5f22-ca0d-4086-b335-1a14e76afcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.201423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbe070-2e4e-42da-bbaf-112ef80ff417 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:22.210650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a2ab5f22-ca0d-4086-b335-1a14e76afcbf None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:22.211914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a2ab5f22-ca0d-4086-b335-1a14e76afcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:22.238485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b78c81-488c-4e57-80d3-b4ffe665dfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:22.239271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b78c81-488c-4e57-80d3-b4ffe665dfac None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.239646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b78c81-488c-4e57-80d3-b4ffe665dfac None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.240037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b78c81-488c-4e57-80d3-b4ffe665dfac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.248134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b78c81-488c-4e57-80d3-b4ffe665dfac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:22.249839 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b78c81-488c-4e57-80d3-b4ffe665dfac None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-26b78c81-488c-4e57-80d3-b4ffe665dfac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:22.265847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:22.267486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:22.269665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:22.269665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:22.269665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:22.269665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:22.269665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:22.269665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:22.269665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:22.269665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:22.269665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:22.269665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:22.279804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 314/623] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Aug 6 01:05:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:22.279804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b78c81-488c-4e57-80d3-b4ffe665dfac None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:22.295041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 310/624] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:22.296076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab5f22-ca0d-4086-b335-1a14e76afcbf None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:22.315698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:22.389413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:22.414174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab5f22-ca0d-4086-b335-1a14e76afcbf None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:22.416166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b78c81-488c-4e57-80d3-b4ffe665dfac None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:22.451557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcdf51-6596-435c-affa-f3899e69974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:22.452399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcdf51-6596-435c-affa-f3899e69974d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.452681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcdf51-6596-435c-affa-f3899e69974d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.453007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcdf51-6596-435c-affa-f3899e69974d None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.453901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcdf51-6596-435c-affa-f3899e69974d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:22.454211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcdf51-6596-435c-affa-f3899e69974d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:22.455796 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcdf51-6596-435c-affa-f3899e69974d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:22.460565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ee74a-1662-41aa-b4da-b5620eb9afa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:22.461402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ee74a-1662-41aa-b4da-b5620eb9afa2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.461852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ee74a-1662-41aa-b4da-b5620eb9afa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.462556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ee74a-1662-41aa-b4da-b5620eb9afa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.475725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ee74a-1662-41aa-b4da-b5620eb9afa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:22.477499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ee74a-1662-41aa-b4da-b5620eb9afa2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:22.478744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 311/625] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Aug 6 01:05:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:22.479603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcdf51-6596-435c-affa-f3899e69974d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bd3ee74a-1662-41aa-b4da-b5620eb9afa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:22.500253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:22.501394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:22.502502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:22.502502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:22.502502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:22.502502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:22.504868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 315/626] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:22.505903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ee74a-1662-41aa-b4da-b5620eb9afa2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:22.573306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:22.596930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcdf51-6596-435c-affa-f3899e69974d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:22.633873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daae9b-ff30-4af8-9d25-8d444cc04e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:22.634661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daae9b-ff30-4af8-9d25-8d444cc04e1e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.635043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daae9b-ff30-4af8-9d25-8d444cc04e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.635413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daae9b-ff30-4af8-9d25-8d444cc04e1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.644978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ee74a-1662-41aa-b4da-b5620eb9afa2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:22.646559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66daae9b-ff30-4af8-9d25-8d444cc04e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-66daae9b-ff30-4af8-9d25-8d444cc04e1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:22.651604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:22.652821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:22.653762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:22.659093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 312/627] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:22.660006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daae9b-ff30-4af8-9d25-8d444cc04e1e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:22.681613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:22.688611 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759dea4-d6d0-4023-9771-828b82805129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:22.689686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759dea4-d6d0-4023-9771-828b82805129 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.706292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759dea4-d6d0-4023-9771-828b82805129 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.706292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759dea4-d6d0-4023-9771-828b82805129 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.771194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daae9b-ff30-4af8-9d25-8d444cc04e1e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:22.783826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759dea4-d6d0-4023-9771-828b82805129 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:22.786196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759dea4-d6d0-4023-9771-828b82805129 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:22.809781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32a868-c203-4952-bd00-4a2eaf2b8dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:22.810996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32a868-c203-4952-bd00-4a2eaf2b8dae None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.811228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32a868-c203-4952-bd00-4a2eaf2b8dae None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.811609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32a868-c203-4952-bd00-4a2eaf2b8dae None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.812428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32a868-c203-4952-bd00-4a2eaf2b8dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:22.812830 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32a868-c203-4952-bd00-4a2eaf2b8dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:22.814301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32a868-c203-4952-bd00-4a2eaf2b8dae None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:22.817511 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:22.835732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 313/628] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:22.835732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32a868-c203-4952-bd00-4a2eaf2b8dae None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:22.911561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 316/629] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:22.911561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759dea4-d6d0-4023-9771-828b82805129 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:22.944829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:22.953384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32a868-c203-4952-bd00-4a2eaf2b8dae None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:22.989552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f8fa6-e5c5-4861-8a0a-a2468be83176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:22.990088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f8fa6-e5c5-4861-8a0a-a2468be83176 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:22.990367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f8fa6-e5c5-4861-8a0a-a2468be83176 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:22.990640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f8fa6-e5c5-4861-8a0a-a2468be83176 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:22.992031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f8fa6-e5c5-4861-8a0a-a2468be83176 None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:22.992338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f8fa6-e5c5-4861-8a0a-a2468be83176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:22.993719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f8fa6-e5c5-4861-8a0a-a2468be83176 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:23.002732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 314/630] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:23.003712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaef87b-6a06-465a-815b-72fee61180f9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.004029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaef87b-6a06-465a-815b-72fee61180f9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.004301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaef87b-6a06-465a-815b-72fee61180f9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.009731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 315/631] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:23.009731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaef87b-6a06-465a-815b-72fee61180f9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:23.025048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759dea4-d6d0-4023-9771-828b82805129 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:23.062564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c78c1-b734-4656-9fe7-3f70d608c292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:23.063840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c78c1-b734-4656-9fe7-3f70d608c292 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.064335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c78c1-b734-4656-9fe7-3f70d608c292 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.064837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c78c1-b734-4656-9fe7-3f70d608c292 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.071194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:23.074463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c78c1-b734-4656-9fe7-3f70d608c292 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:23.076583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c78c1-b734-4656-9fe7-3f70d608c292 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-848c78c1-b734-4656-9fe7-3f70d608c292 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:23.090419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:23.091391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:23.096847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:23.096847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:23.096847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:23.096847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:23.096847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:23.096847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:23.096847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:23.096847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:23.096847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 317/632] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:23.096847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c78c1-b734-4656-9fe7-3f70d608c292 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:23.125670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaef87b-6a06-465a-815b-72fee61180f9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:23.161269 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a8ff8-dae2-422d-b541-fb275d1f420f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:23.161900 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a8ff8-dae2-422d-b541-fb275d1f420f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.162177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a8ff8-dae2-422d-b541-fb275d1f420f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.162448 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a8ff8-dae2-422d-b541-fb275d1f420f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.174325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a8ff8-dae2-422d-b541-fb275d1f420f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:23.175799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a8ff8-dae2-422d-b541-fb275d1f420f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-459a8ff8-dae2-422d-b541-fb275d1f420f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:23.191012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:23.191997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:23.192904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:23.192904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:23.192904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:23.192904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:23.192904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 316/633] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:23.193609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d88f59-4950-4b2a-92bc-ef219f77fc24 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.193609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d88f59-4950-4b2a-92bc-ef219f77fc24 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.193842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d88f59-4950-4b2a-92bc-ef219f77fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.199000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:23.208673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c78c1-b734-4656-9fe7-3f70d608c292 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:23.212612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-55d88f59-4950-4b2a-92bc-ef219f77fc24 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:23.213786 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-55d88f59-4950-4b2a-92bc-ef219f77fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:23.246601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef0f5-e35e-44ad-95a8-1cef1ae31441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:23.247165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef0f5-e35e-44ad-95a8-1cef1ae31441 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.247446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef0f5-e35e-44ad-95a8-1cef1ae31441 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.247727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ef0f5-e35e-44ad-95a8-1cef1ae31441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.258003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ef0f5-e35e-44ad-95a8-1cef1ae31441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:23.259450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ef0f5-e35e-44ad-95a8-1cef1ae31441 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-185ef0f5-e35e-44ad-95a8-1cef1ae31441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:23.274480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:23.275448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:23.276730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:23.276730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:23.276730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:23.276730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:23.276730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 318/634] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:23.276730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef0f5-e35e-44ad-95a8-1cef1ae31441 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:23.295952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 317/635] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:23.295952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d88f59-4950-4b2a-92bc-ef219f77fc24 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:23.342450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:23.387200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ef0f5-e35e-44ad-95a8-1cef1ae31441 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:23.402749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d88f59-4950-4b2a-92bc-ef219f77fc24 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:23.421202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80c906-89ff-4db7-be48-df0026546af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:23.421811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80c906-89ff-4db7-be48-df0026546af8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.422090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80c906-89ff-4db7-be48-df0026546af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.422364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80c906-89ff-4db7-be48-df0026546af8 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.423102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80c906-89ff-4db7-be48-df0026546af8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:23.423405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80c906-89ff-4db7-be48-df0026546af8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:23.424862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80c906-89ff-4db7-be48-df0026546af8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:23.435507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8d72e-8f6e-44ab-9ee0-b03cfaa6c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:23.436156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8d72e-8f6e-44ab-9ee0-b03cfaa6c835 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.436441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8d72e-8f6e-44ab-9ee0-b03cfaa6c835 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.436562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 319/636] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:23.436721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8d72e-8f6e-44ab-9ee0-b03cfaa6c835 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.437210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80c906-89ff-4db7-be48-df0026546af8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:23.437394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8d72e-8f6e-44ab-9ee0-b03cfaa6c835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:23.437697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8d72e-8f6e-44ab-9ee0-b03cfaa6c835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:23.439054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8d72e-8f6e-44ab-9ee0-b03cfaa6c835 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:23.442352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:23.450466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 318/637] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:23.451270 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8d72e-8f6e-44ab-9ee0-b03cfaa6c835 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:23.534943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80c906-89ff-4db7-be48-df0026546af8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:23.548672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:23.551511 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8d72e-8f6e-44ab-9ee0-b03cfaa6c835 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:23.567569 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a33801-fe46-4695-b326-b53ce1dd37cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:23.568209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a33801-fe46-4695-b326-b53ce1dd37cc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.568491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a33801-fe46-4695-b326-b53ce1dd37cc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.568768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a33801-fe46-4695-b326-b53ce1dd37cc None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.570000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a33801-fe46-4695-b326-b53ce1dd37cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:23.570307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a33801-fe46-4695-b326-b53ce1dd37cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:23.571658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a33801-fe46-4695-b326-b53ce1dd37cc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:23.588668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 320/638] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:23.588668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8cfc6-16b8-4c56-8a39-92e3aff7e20d None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.588668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8cfc6-16b8-4c56-8a39-92e3aff7e20d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.588668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8cfc6-16b8-4c56-8a39-92e3aff7e20d None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.588668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 321/639] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:23.588668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8cfc6-16b8-4c56-8a39-92e3aff7e20d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:23.590746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cbf51-91a3-4aa3-98dc-01a75cbc1a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:23.591397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cbf51-91a3-4aa3-98dc-01a75cbc1a86 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.591684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cbf51-91a3-4aa3-98dc-01a75cbc1a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.591991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cbf51-91a3-4aa3-98dc-01a75cbc1a86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.602542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362cbf51-91a3-4aa3-98dc-01a75cbc1a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-362cbf51-91a3-4aa3-98dc-01a75cbc1a86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:23.607577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:23.608537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:23.609387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 319/640] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:23.610132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cbf51-91a3-4aa3-98dc-01a75cbc1a86 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:23.654899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:23.692264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8cfc6-16b8-4c56-8a39-92e3aff7e20d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:23.714198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cbf51-91a3-4aa3-98dc-01a75cbc1a86 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:23.725847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8dc76-f7f6-4cc6-bfd7-097e018df571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:23.726509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8dc76-f7f6-4cc6-bfd7-097e018df571 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.726790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8dc76-f7f6-4cc6-bfd7-097e018df571 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.727061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8dc76-f7f6-4cc6-bfd7-097e018df571 None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/33aa626411f1474595102a4541af901a/roles/4dabc11cde5340ee9eb2c28e6154b3ec` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.740698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8dc76-f7f6-4cc6-bfd7-097e018df571 None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=33aa626411f1474595102a4541af901a, role_id=4dabc11cde5340ee9eb2c28e6154b3ec)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:23.742129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8dc76-f7f6-4cc6-bfd7-097e018df571 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:23.746598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a7032-6c01-43bf-9943-13d679fc5581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:23.747237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a7032-6c01-43bf-9943-13d679fc5581 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.747518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a7032-6c01-43bf-9943-13d679fc5581 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.747804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a7032-6c01-43bf-9943-13d679fc5581 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.754216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a7032-6c01-43bf-9943-13d679fc5581 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:23.755637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a7032-6c01-43bf-9943-13d679fc5581 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-f64a7032-6c01-43bf-9943-13d679fc5581 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:23.767021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:23.775740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:23.776889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:23.776889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:23.776889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:23.776889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:23.776889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:23.776889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:23.776889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:23.776889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:23.776889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 320/641] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:23.776889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a7032-6c01-43bf-9943-13d679fc5581 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:23.791618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:23.830253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 322/642] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:05:23 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/33aa626411f1474595102a4541af901a/roles/4dabc11cde5340ee9eb2c28e6154b3ec => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:05:23.831489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee506f-5632-46a9-9a7d-643783d99842 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.831799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee506f-5632-46a9-9a7d-643783d99842 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.832079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee506f-5632-46a9-9a7d-643783d99842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.860869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2dee506f-5632-46a9-9a7d-643783d99842 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:23.862013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2dee506f-5632-46a9-9a7d-643783d99842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:23.875076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a7032-6c01-43bf-9943-13d679fc5581 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:23.914083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7343456-1b31-4894-8147-98fd7be381d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:23.914778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7343456-1b31-4894-8147-98fd7be381d4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:23.915059 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7343456-1b31-4894-8147-98fd7be381d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:23.915328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7343456-1b31-4894-8147-98fd7be381d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:23.924049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7343456-1b31-4894-8147-98fd7be381d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:23.925538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7343456-1b31-4894-8147-98fd7be381d4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b7343456-1b31-4894-8147-98fd7be381d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:23.938653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:23.939683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 321/644] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:23.949986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7343456-1b31-4894-8147-98fd7be381d4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:23.950909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 323/644] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:23.950909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee506f-5632-46a9-9a7d-643783d99842 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:24.014458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:24.049572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:24.068527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7343456-1b31-4894-8147-98fd7be381d4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.068818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee506f-5632-46a9-9a7d-643783d99842 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.102350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ca326-7cea-4682-80e3-5a023a0bec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:24.103063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ca326-7cea-4682-80e3-5a023a0bec52 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:24.103249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ca326-7cea-4682-80e3-5a023a0bec52 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:24.103522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ca326-7cea-4682-80e3-5a023a0bec52 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:24.104276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ca326-7cea-4682-80e3-5a023a0bec52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:24.104578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ca326-7cea-4682-80e3-5a023a0bec52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:24.105823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a5f13-07b0-424e-b518-f895cdad475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:24.106020 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ca326-7cea-4682-80e3-5a023a0bec52 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:24.106548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a5f13-07b0-424e-b518-f895cdad475b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:24.106840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a5f13-07b0-424e-b518-f895cdad475b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:24.107114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a5f13-07b0-424e-b518-f895cdad475b None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:24.107896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a5f13-07b0-424e-b518-f895cdad475b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:24.108207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a5f13-07b0-424e-b518-f895cdad475b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:24.109624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a5f13-07b0-424e-b518-f895cdad475b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:24.120446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 322/645] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:24.120446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ca326-7cea-4682-80e3-5a023a0bec52 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:24.130742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 324/646] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:24.130742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a5f13-07b0-424e-b518-f895cdad475b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:24.226337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ca326-7cea-4682-80e3-5a023a0bec52 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.232663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a5f13-07b0-424e-b518-f895cdad475b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.235983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:24.262931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd31b84-be77-4cea-8048-62af97737f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:24.263557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd31b84-be77-4cea-8048-62af97737f3f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:24.263865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd31b84-be77-4cea-8048-62af97737f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:24.264298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd31b84-be77-4cea-8048-62af97737f3f None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:24.264924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd31b84-be77-4cea-8048-62af97737f3f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:24.265232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd31b84-be77-4cea-8048-62af97737f3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:24.266757 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd31b84-be77-4cea-8048-62af97737f3f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:24.270110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b12e82-5573-4c07-a794-4c427725beb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:24.271024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12e82-5573-4c07-a794-4c427725beb3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:24.271743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12e82-5573-4c07-a794-4c427725beb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:24.272053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12e82-5573-4c07-a794-4c427725beb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:24.281080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:24.282487 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b12e82-5573-4c07-a794-4c427725beb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:24.289663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 323/647] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Aug 6 01:05:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:24.289663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd31b84-be77-4cea-8048-62af97737f3f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-37b12e82-5573-4c07-a794-4c427725beb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:24.299658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:24.300941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 325/648] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:24.301774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b12e82-5573-4c07-a794-4c427725beb3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:24.398307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b12e82-5573-4c07-a794-4c427725beb3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.401375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd31b84-be77-4cea-8048-62af97737f3f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.439724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12899890-7063-4de8-b4d2-c92dcb4bf400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:24.440372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12899890-7063-4de8-b4d2-c92dcb4bf400 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:24.440669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12899890-7063-4de8-b4d2-c92dcb4bf400 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:24.440833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4904d-8b51-4dbe-8924-60cf4ea71c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:24.440942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12899890-7063-4de8-b4d2-c92dcb4bf400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:24.441437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4904d-8b51-4dbe-8924-60cf4ea71c91 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:24.441708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12899890-7063-4de8-b4d2-c92dcb4bf400 None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=db1b205d165543d0a3a69a2db6af5d1c) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:24.441805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4904d-8b51-4dbe-8924-60cf4ea71c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:24.441984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4904d-8b51-4dbe-8924-60cf4ea71c91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:24.442078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12899890-7063-4de8-b4d2-c92dcb4bf400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:24.443445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12899890-7063-4de8-b4d2-c92dcb4bf400 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:24.460046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 326/649] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:24 2022] GET /identity/v3/role_assignments?user.id=db1b205d165543d0a3a69a2db6af5d1c&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:24.460046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12899890-7063-4de8-b4d2-c92dcb4bf400 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:24.460314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4904d-8b51-4dbe-8924-60cf4ea71c91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:24.460314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4904d-8b51-4dbe-8924-60cf4ea71c91 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-23b4904d-8b51-4dbe-8924-60cf4ea71c91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:24.477669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:24.478838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:24.479964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:24.479964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:24.479964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:24.479964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:24.479964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 324/650] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:24.480243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4904d-8b51-4dbe-8924-60cf4ea71c91 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:24.513199 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:24.542774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:24.605046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4904d-8b51-4dbe-8924-60cf4ea71c91 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.605346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12899890-7063-4de8-b4d2-c92dcb4bf400 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.645585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5379283-cdc1-4e7a-9bb4-f9e4fe5221cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:24.646271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5379283-cdc1-4e7a-9bb4-f9e4fe5221cc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:24.646557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5379283-cdc1-4e7a-9bb4-f9e4fe5221cc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:24.646834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5379283-cdc1-4e7a-9bb4-f9e4fe5221cc None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:24.647119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3620-f553-4538-9fc0-532bea16e9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:24.647595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5379283-cdc1-4e7a-9bb4-f9e4fe5221cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:24.647794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3620-f553-4538-9fc0-532bea16e9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:24.647964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5379283-cdc1-4e7a-9bb4-f9e4fe5221cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:24.648112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3620-f553-4538-9fc0-532bea16e9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:24.648360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3620-f553-4538-9fc0-532bea16e9c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:24.649332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5379283-cdc1-4e7a-9bb4-f9e4fe5221cc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:24.659240 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 325/651] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:24.660483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5379283-cdc1-4e7a-9bb4-f9e4fe5221cc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:24.737854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc3620-f553-4538-9fc0-532bea16e9c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:24.738739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc3620-f553-4538-9fc0-532bea16e9c1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:24.775120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:24.780871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5379283-cdc1-4e7a-9bb4-f9e4fe5221cc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.798574 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:24.824293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5c9b0-2d9f-4ca8-9897-de40fb9ea08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:24.825646 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5c9b0-2d9f-4ca8-9897-de40fb9ea08b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:24.826023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5c9b0-2d9f-4ca8-9897-de40fb9ea08b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:24.826392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5c9b0-2d9f-4ca8-9897-de40fb9ea08b None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:24.827715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5c9b0-2d9f-4ca8-9897-de40fb9ea08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:24.828149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5c9b0-2d9f-4ca8-9897-de40fb9ea08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:24.830728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5c9b0-2d9f-4ca8-9897-de40fb9ea08b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:24.848009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 326/652] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:24.849161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5c9b0-2d9f-4ca8-9897-de40fb9ea08b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:24.873245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 327/653] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:24.874240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3620-f553-4538-9fc0-532bea16e9c1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:24.959344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5c9b0-2d9f-4ca8-9897-de40fb9ea08b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.978320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3620-f553-4538-9fc0-532bea16e9c1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:24.996498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a84017-ed78-4f84-9625-72fe9394d06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:24.997541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a84017-ed78-4f84-9625-72fe9394d06c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:24.998026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a84017-ed78-4f84-9625-72fe9394d06c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:24.998483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a84017-ed78-4f84-9625-72fe9394d06c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:25.005160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:25.009864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a84017-ed78-4f84-9625-72fe9394d06c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:25.011583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a84017-ed78-4f84-9625-72fe9394d06c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:25.013249 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb04ab5-899e-41ed-be44-d754243c69f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:25.013932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04ab5-899e-41ed-be44-d754243c69f5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:25.014334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04ab5-899e-41ed-be44-d754243c69f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:25.014439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04ab5-899e-41ed-be44-d754243c69f5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:25.024205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb04ab5-899e-41ed-be44-d754243c69f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:25.025690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb04ab5-899e-41ed-be44-d754243c69f5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-93a84017-ed78-4f84-9625-72fe9394d06c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:25.031250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:25.032301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:25.033217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:25.033217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:25.033217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:25.033217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:25.035542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 327/654] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:25.036619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a84017-ed78-4f84-9625-72fe9394d06c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8bb04ab5-899e-41ed-be44-d754243c69f5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:25.038689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:25.039612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:25.040781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:25.040781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:25.040781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:25.040781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:25.040781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:25.040781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:25.040781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:25.040781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:25.040781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 01:05:25.040781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:25.044947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 328/655] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:25.047077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb04ab5-899e-41ed-be44-d754243c69f5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:25.077473 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:25.167493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb04ab5-899e-41ed-be44-d754243c69f5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:25.178438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a84017-ed78-4f84-9625-72fe9394d06c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:25.204641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7cdad-e4ff-4e05-a504-3b1f9c13abfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:25.205259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7cdad-e4ff-4e05-a504-3b1f9c13abfc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:25.205544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7cdad-e4ff-4e05-a504-3b1f9c13abfc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:25.205820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7cdad-e4ff-4e05-a504-3b1f9c13abfc None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:25.206513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7cdad-e4ff-4e05-a504-3b1f9c13abfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:25.206819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7cdad-e4ff-4e05-a504-3b1f9c13abfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:25.208232 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7cdad-e4ff-4e05-a504-3b1f9c13abfc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:25.225008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327851f0-d686-42fb-b488-7b44802deb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:25.225760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 329/656] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:25.226254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327851f0-d686-42fb-b488-7b44802deb7f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:25.226604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7cdad-e4ff-4e05-a504-3b1f9c13abfc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:25.226985 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327851f0-d686-42fb-b488-7b44802deb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:25.227457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327851f0-d686-42fb-b488-7b44802deb7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:25.241534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327851f0-d686-42fb-b488-7b44802deb7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:25.244169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327851f0-d686-42fb-b488-7b44802deb7f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-327851f0-d686-42fb-b488-7b44802deb7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:25.266661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:25.267602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:25.268552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:25.268552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:25.268552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:25.268552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:25.271280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 328/657] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:25.272508 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327851f0-d686-42fb-b488-7b44802deb7f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:25.281858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:25.336806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7cdad-e4ff-4e05-a504-3b1f9c13abfc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:25.375234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d028fc-624a-44d2-bbdb-7d24e4a2f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:25.375618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d028fc-624a-44d2-bbdb-7d24e4a2f774 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:25.375954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d028fc-624a-44d2-bbdb-7d24e4a2f774 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:25.376193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d028fc-624a-44d2-bbdb-7d24e4a2f774 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:25.377005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d028fc-624a-44d2-bbdb-7d24e4a2f774 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:25.377315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d028fc-624a-44d2-bbdb-7d24e4a2f774 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:25.378770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d028fc-624a-44d2-bbdb-7d24e4a2f774 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:25.395771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:25.401215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 330/658] 10.222.0.25 () {56 vars in 1137 bytes} [Sat Aug 6 01:05:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:25.402136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d028fc-624a-44d2-bbdb-7d24e4a2f774 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:25.429526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327851f0-d686-42fb-b488-7b44802deb7f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:25.465258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc133869-55b4-486f-9dcb-9100a5b92421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:25.466315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc133869-55b4-486f-9dcb-9100a5b92421 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:25.466797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc133869-55b4-486f-9dcb-9100a5b92421 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:25.467265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc133869-55b4-486f-9dcb-9100a5b92421 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:25.475112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc133869-55b4-486f-9dcb-9100a5b92421 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:25.476936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc133869-55b4-486f-9dcb-9100a5b92421 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-dc133869-55b4-486f-9dcb-9100a5b92421 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:25.488498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:25.489475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:25.490454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:25.490454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:25.490454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:25.490454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:25.490454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:25.490454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:25.490454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:25.490454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:25.490454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 01:05:25.490454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:25.493296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 329/659] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:25.494271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc133869-55b4-486f-9dcb-9100a5b92421 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:25.516649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d028fc-624a-44d2-bbdb-7d24e4a2f774 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:25.527073 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:25.557215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36539b-46d6-4b9d-990b-e5b54a51b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:25.557839 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36539b-46d6-4b9d-990b-e5b54a51b05a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:25.558280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36539b-46d6-4b9d-990b-e5b54a51b05a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:25.558594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36539b-46d6-4b9d-990b-e5b54a51b05a None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:25.559342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36539b-46d6-4b9d-990b-e5b54a51b05a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:25.559818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36539b-46d6-4b9d-990b-e5b54a51b05a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:25.562348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36539b-46d6-4b9d-990b-e5b54a51b05a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:25.575773 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 331/660] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:25.582295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36539b-46d6-4b9d-990b-e5b54a51b05a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:25.606156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc133869-55b4-486f-9dcb-9100a5b92421 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:25.641985 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7e951-fa30-4b91-aed9-35f7ab10b9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:25.642993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7e951-fa30-4b91-aed9-35f7ab10b9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:25.643479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7e951-fa30-4b91-aed9-35f7ab10b9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:25.644003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7e951-fa30-4b91-aed9-35f7ab10b9d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:25.653236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:25.701755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36539b-46d6-4b9d-990b-e5b54a51b05a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:25.724331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7e951-fa30-4b91-aed9-35f7ab10b9d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:25.726112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7e951-fa30-4b91-aed9-35f7ab10b9d2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:25.742988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35479c53-2275-43ec-9e05-234f946901e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:25.743911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35479c53-2275-43ec-9e05-234f946901e6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:25.744359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35479c53-2275-43ec-9e05-234f946901e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:25.744780 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35479c53-2275-43ec-9e05-234f946901e6 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:25.745839 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35479c53-2275-43ec-9e05-234f946901e6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:25.746294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35479c53-2275-43ec-9e05-234f946901e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:25.748081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35479c53-2275-43ec-9e05-234f946901e6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:25.772319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 332/661] 10.222.0.25 () {56 vars in 1139 bytes} [Sat Aug 6 01:05:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:25.773562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35479c53-2275-43ec-9e05-234f946901e6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:25.800818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:25.831261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 330/662] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:25.832279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7e951-fa30-4b91-aed9-35f7ab10b9d2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:25.872188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:25.885962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35479c53-2275-43ec-9e05-234f946901e6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:25.923436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960ba80-f15f-4509-8d10-617de5d9ecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:25.924396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960ba80-f15f-4509-8d10-617de5d9ecb9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:25.924825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960ba80-f15f-4509-8d10-617de5d9ecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:25.925247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960ba80-f15f-4509-8d10-617de5d9ecb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:25.932762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7e951-fa30-4b91-aed9-35f7ab10b9d2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:25.933108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960ba80-f15f-4509-8d10-617de5d9ecb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:25.934864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960ba80-f15f-4509-8d10-617de5d9ecb9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0960ba80-f15f-4509-8d10-617de5d9ecb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:25.947761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:25.948825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:25.949922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:25.949922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:25.949922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:25.949922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:25.949922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:25.949922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:25.949922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:25.949922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:25.952035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 333/663] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:25.953302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960ba80-f15f-4509-8d10-617de5d9ecb9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:25.971629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d5eb4-e576-4dd0-94cf-167bde9d98a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:25.972710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d5eb4-e576-4dd0-94cf-167bde9d98a2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:25.973090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d5eb4-e576-4dd0-94cf-167bde9d98a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:25.973488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d5eb4-e576-4dd0-94cf-167bde9d98a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.043765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:26.047374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d5eb4-e576-4dd0-94cf-167bde9d98a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:26.048936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d5eb4-e576-4dd0-94cf-167bde9d98a2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:26.060895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960ba80-f15f-4509-8d10-617de5d9ecb9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:26.088803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:26.097822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6515b8-5b8b-4bf7-a87f-790583c38639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:26.098557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6515b8-5b8b-4bf7-a87f-790583c38639 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.098898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6515b8-5b8b-4bf7-a87f-790583c38639 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.099261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6515b8-5b8b-4bf7-a87f-790583c38639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.111213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6515b8-5b8b-4bf7-a87f-790583c38639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:26.112777 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6515b8-5b8b-4bf7-a87f-790583c38639 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9a6515b8-5b8b-4bf7-a87f-790583c38639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:26.127269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:26.128241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:26.134297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:26.134297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:26.134297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:26.134297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:26.134297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 334/664] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:26.134297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6515b8-5b8b-4bf7-a87f-790583c38639 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:26.157580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 331/665] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:26.158867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5daa4-bfcb-49fb-96f3-3e017966dd07 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.159152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5daa4-bfcb-49fb-96f3-3e017966dd07 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.159425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5daa4-bfcb-49fb-96f3-3e017966dd07 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.163913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 332/666] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:26.163913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5daa4-bfcb-49fb-96f3-3e017966dd07 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:26.243042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6515b8-5b8b-4bf7-a87f-790583c38639 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:26.273948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5daa4-bfcb-49fb-96f3-3e017966dd07 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:26.276515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1d856-0398-4522-b50b-58a5aa08dfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:26.277136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1d856-0398-4522-b50b-58a5aa08dfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.277416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1d856-0398-4522-b50b-58a5aa08dfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.277693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1d856-0398-4522-b50b-58a5aa08dfa7 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.278434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1d856-0398-4522-b50b-58a5aa08dfa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:26.278741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1d856-0398-4522-b50b-58a5aa08dfa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:26.280235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1d856-0398-4522-b50b-58a5aa08dfa7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:26.282950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:26.298104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 335/667] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:26.298104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1d856-0398-4522-b50b-58a5aa08dfa7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:26.314513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b42e56-3d33-48da-aac0-6a989e36b2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:26.315313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b42e56-3d33-48da-aac0-6a989e36b2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.315770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b42e56-3d33-48da-aac0-6a989e36b2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.316060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b42e56-3d33-48da-aac0-6a989e36b2c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.316748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b42e56-3d33-48da-aac0-6a989e36b2c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:26.317046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b42e56-3d33-48da-aac0-6a989e36b2c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:26.318497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b42e56-3d33-48da-aac0-6a989e36b2c4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:26.326590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:26.336879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 333/668] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:26.338546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823060b9-7a8b-47a3-b185-3190560c83d2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.338821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823060b9-7a8b-47a3-b185-3190560c83d2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.339099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823060b9-7a8b-47a3-b185-3190560c83d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.357487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-823060b9-7a8b-47a3-b185-3190560c83d2 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:26.358661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-823060b9-7a8b-47a3-b185-3190560c83d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:26.398856 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1d856-0398-4522-b50b-58a5aa08dfa7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:26.433550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 334/669] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:26.434692 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823060b9-7a8b-47a3-b185-3190560c83d2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:26.440778 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d12e-9486-4451-ba95-e91c9abaeeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:26.441612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d12e-9486-4451-ba95-e91c9abaeeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.442177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d12e-9486-4451-ba95-e91c9abaeeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.442602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d12e-9486-4451-ba95-e91c9abaeeb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.454983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76d12e-9486-4451-ba95-e91c9abaeeb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:26.456754 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76d12e-9486-4451-ba95-e91c9abaeeb8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3e76d12e-9486-4451-ba95-e91c9abaeeb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:26.472422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:26.473541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:26.474719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:26.474719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:26.474719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:26.474719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:26.478086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 336/670] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:26.478927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d12e-9486-4451-ba95-e91c9abaeeb8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:26.532198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:26.547407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823060b9-7a8b-47a3-b185-3190560c83d2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:26.575094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:26.588929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5294f-8fef-47fb-9fb5-c4c9c3d232c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:26.589582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d12e-9486-4451-ba95-e91c9abaeeb8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:26.589697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5294f-8fef-47fb-9fb5-c4c9c3d232c8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.590009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5294f-8fef-47fb-9fb5-c4c9c3d232c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.590347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5294f-8fef-47fb-9fb5-c4c9c3d232c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.591253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5294f-8fef-47fb-9fb5-c4c9c3d232c8 None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=ddd0e588d52e40f08a2dc1062ecd4232) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:26.591659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5294f-8fef-47fb-9fb5-c4c9c3d232c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:26.593178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5294f-8fef-47fb-9fb5-c4c9c3d232c8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:26.608058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 335/671] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:26 2022] GET /identity/v3/role_assignments?user.id=ddd0e588d52e40f08a2dc1062ecd4232&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:26.608058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5294f-8fef-47fb-9fb5-c4c9c3d232c8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:26.646454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd3172-f55e-45d0-9f00-f5064116cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:26.647108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd3172-f55e-45d0-9f00-f5064116cdae None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.647387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd3172-f55e-45d0-9f00-f5064116cdae None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.647710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd3172-f55e-45d0-9f00-f5064116cdae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.655336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd3172-f55e-45d0-9f00-f5064116cdae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:26.656869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd3172-f55e-45d0-9f00-f5064116cdae None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-6fcd3172-f55e-45d0-9f00-f5064116cdae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:26.669862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:26.671014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:26.677159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:26.677159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:26.677159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:26.677159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:26.677159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:26.677159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:26.677159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:26.677159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:26.677159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 337/672] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:26.677159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd3172-f55e-45d0-9f00-f5064116cdae None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:26.717522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5294f-8fef-47fb-9fb5-c4c9c3d232c8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:26.755630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916762e-f80f-4a19-b17f-6114de695a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:26.756678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916762e-f80f-4a19-b17f-6114de695a0d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.757087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916762e-f80f-4a19-b17f-6114de695a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.757450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916762e-f80f-4a19-b17f-6114de695a0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.771014 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916762e-f80f-4a19-b17f-6114de695a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d916762e-f80f-4a19-b17f-6114de695a0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:26.776447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:26.777564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 336/673] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:26.781022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebccb98-0c73-440b-99b2-947d9d6b4533 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.781781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebccb98-0c73-440b-99b2-947d9d6b4533 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.781781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebccb98-0c73-440b-99b2-947d9d6b4533 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.781781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 337/674] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:26.781781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebccb98-0c73-440b-99b2-947d9d6b4533 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:26.797347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd3172-f55e-45d0-9f00-f5064116cdae None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:26.808106 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:26.839612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989658f2-ec0b-453f-88c7-95c9d0efcbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:26.840694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989658f2-ec0b-453f-88c7-95c9d0efcbbc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.841399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989658f2-ec0b-453f-88c7-95c9d0efcbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.841766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989658f2-ec0b-453f-88c7-95c9d0efcbbc None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.842406 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:26.842651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989658f2-ec0b-453f-88c7-95c9d0efcbbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:26.843069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989658f2-ec0b-453f-88c7-95c9d0efcbbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:26.844592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989658f2-ec0b-453f-88c7-95c9d0efcbbc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:26.858098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 338/675] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:26.859051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989658f2-ec0b-453f-88c7-95c9d0efcbbc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:26.897795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebccb98-0c73-440b-99b2-947d9d6b4533 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:26.935014 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dd22f-fa21-44c4-84f0-0f2f65831d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:26.935652 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dd22f-fa21-44c4-84f0-0f2f65831d02 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.935970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dd22f-fa21-44c4-84f0-0f2f65831d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.936252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dd22f-fa21-44c4-84f0-0f2f65831d02 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.937007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dd22f-fa21-44c4-84f0-0f2f65831d02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:26.937319 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dd22f-fa21-44c4-84f0-0f2f65831d02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:26.938781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dd22f-fa21-44c4-84f0-0f2f65831d02 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:26.954134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 338/676] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:26.955350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea1e56-14d2-4930-80ff-4966c53dbbf4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:26.955639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea1e56-14d2-4930-80ff-4966c53dbbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:26.955940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea1e56-14d2-4930-80ff-4966c53dbbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:26.969794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989658f2-ec0b-453f-88c7-95c9d0efcbbc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:26.974136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a5ea1e56-14d2-4930-80ff-4966c53dbbf4 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:26.975333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a5ea1e56-14d2-4930-80ff-4966c53dbbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:27.006624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41af3e-ba36-4bcf-ab6e-9d01d644f930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:27.007265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41af3e-ba36-4bcf-ab6e-9d01d644f930 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:27.007574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41af3e-ba36-4bcf-ab6e-9d01d644f930 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:27.007843 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41af3e-ba36-4bcf-ab6e-9d01d644f930 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:27.008566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41af3e-ba36-4bcf-ab6e-9d01d644f930 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:27.008879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41af3e-ba36-4bcf-ab6e-9d01d644f930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:27.010254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41af3e-ba36-4bcf-ab6e-9d01d644f930 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:27.023389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 339/677] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:27.023389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41af3e-ba36-4bcf-ab6e-9d01d644f930 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:27.056097 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 339/678] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:27.056097 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea1e56-14d2-4930-80ff-4966c53dbbf4 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:27.062639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:27.104896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:27.143202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41af3e-ba36-4bcf-ab6e-9d01d644f930 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:27.169065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea1e56-14d2-4930-80ff-4966c53dbbf4 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:27.180182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197b681-1321-4c2c-82ca-07ce3efd0290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:27.180829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197b681-1321-4c2c-82ca-07ce3efd0290 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:27.181109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197b681-1321-4c2c-82ca-07ce3efd0290 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:27.181386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197b681-1321-4c2c-82ca-07ce3efd0290 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:27.187995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197b681-1321-4c2c-82ca-07ce3efd0290 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:27.189430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197b681-1321-4c2c-82ca-07ce3efd0290 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3197b681-1321-4c2c-82ca-07ce3efd0290 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:27.200473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:27.201363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:27.202556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:27.202556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:27.202556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:27.202556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:27.202556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:27.202556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:27.202556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:27.202556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:27.204102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1c507-2c9f-4698-b8cf-42a74cdf6db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:27.204721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1c507-2c9f-4698-b8cf-42a74cdf6db9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:27.205002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1c507-2c9f-4698-b8cf-42a74cdf6db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:27.205273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1c507-2c9f-4698-b8cf-42a74cdf6db9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:27.206070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1c507-2c9f-4698-b8cf-42a74cdf6db9 None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=9a234a96eccd4f38b0213d7d0461ad24) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:27.206519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1c507-2c9f-4698-b8cf-42a74cdf6db9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:27.206612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 340/679] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:27.207325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197b681-1321-4c2c-82ca-07ce3efd0290 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:27.207973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1c507-2c9f-4698-b8cf-42a74cdf6db9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:27.224117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 340/680] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/role_assignments?user.id=9a234a96eccd4f38b0213d7d0461ad24&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:27.224117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1c507-2c9f-4698-b8cf-42a74cdf6db9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:27.338998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:27.342120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1c507-2c9f-4698-b8cf-42a74cdf6db9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:27.350261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197b681-1321-4c2c-82ca-07ce3efd0290 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:27.351138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:27.390045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6d22d-1876-48f9-b898-9ef0fbdacd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:27.390707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d22d-1876-48f9-b898-9ef0fbdacd02 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:27.391143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d22d-1876-48f9-b898-9ef0fbdacd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:27.391413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d22d-1876-48f9-b898-9ef0fbdacd02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:27.396381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cbca75-69f7-4f0a-94bf-4e5ff0a0c3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:27.397077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbca75-69f7-4f0a-94bf-4e5ff0a0c3cb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:27.397306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbca75-69f7-4f0a-94bf-4e5ff0a0c3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:27.398238 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbca75-69f7-4f0a-94bf-4e5ff0a0c3cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:27.408910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6d22d-1876-48f9-b898-9ef0fbdacd02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-76f6d22d-1876-48f9-b898-9ef0fbdacd02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:27.414433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:27.415362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:27.417767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:27.418716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cbca75-69f7-4f0a-94bf-4e5ff0a0c3cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:27.418812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 341/681] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:27.418812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6d22d-1876-48f9-b898-9ef0fbdacd02 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:27.418969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cbca75-69f7-4f0a-94bf-4e5ff0a0c3cb None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-64cbca75-69f7-4f0a-94bf-4e5ff0a0c3cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:27.437058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:27.438219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:27.439200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:27.439200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:27.439200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:27.439200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:27.439764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 341/682] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:27.442107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cbca75-69f7-4f0a-94bf-4e5ff0a0c3cb None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:27.542564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6d22d-1876-48f9-b898-9ef0fbdacd02 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:27.554678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cbca75-69f7-4f0a-94bf-4e5ff0a0c3cb None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:27.580189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccf1a0-d886-48e3-8d7f-355eb1df6cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:27.581092 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf1a0-d886-48e3-8d7f-355eb1df6cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:27.581632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf1a0-d886-48e3-8d7f-355eb1df6cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:27.582049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf1a0-d886-48e3-8d7f-355eb1df6cd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:27.583022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccf1a0-d886-48e3-8d7f-355eb1df6cd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:27.583474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccf1a0-d886-48e3-8d7f-355eb1df6cd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:27.585239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccf1a0-d886-48e3-8d7f-355eb1df6cd3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:27.591868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce87c38-33c4-498a-860c-54c83eae8516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:27.592559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce87c38-33c4-498a-860c-54c83eae8516 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:27.592937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce87c38-33c4-498a-860c-54c83eae8516 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:27.593306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce87c38-33c4-498a-860c-54c83eae8516 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:27.594186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce87c38-33c4-498a-860c-54c83eae8516 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:27.594526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce87c38-33c4-498a-860c-54c83eae8516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:27.596321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce87c38-33c4-498a-860c-54c83eae8516 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:27.599974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 342/683] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:27.601525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccf1a0-d886-48e3-8d7f-355eb1df6cd3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:27.610112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 342/684] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:27.611105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce87c38-33c4-498a-860c-54c83eae8516 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:27.614876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:27.652186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:27.713538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccf1a0-d886-48e3-8d7f-355eb1df6cd3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:27.723656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce87c38-33c4-498a-860c-54c83eae8516 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:27.753950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ea6eb-cacc-4b50-a5e1-02b148f24a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:27.754584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ea6eb-cacc-4b50-a5e1-02b148f24a84 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:27.755011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ea6eb-cacc-4b50-a5e1-02b148f24a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:27.755306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ea6eb-cacc-4b50-a5e1-02b148f24a84 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:27.756358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ea6eb-cacc-4b50-a5e1-02b148f24a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:27.756673 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ea6eb-cacc-4b50-a5e1-02b148f24a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:27.758507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ea6eb-cacc-4b50-a5e1-02b148f24a84 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:27.767978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8b79f-e09c-402b-ab82-c83097b22388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:27.769269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8b79f-e09c-402b-ab82-c83097b22388 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:27.769649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8b79f-e09c-402b-ab82-c83097b22388 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:27.770171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8b79f-e09c-402b-ab82-c83097b22388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:27.771161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8b79f-e09c-402b-ab82-c83097b22388 None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=2c14fb133a0f419390fb0d00c600bdd4) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:27.771574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8b79f-e09c-402b-ab82-c83097b22388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:27.772981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 343/685] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:27.773925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ea6eb-cacc-4b50-a5e1-02b148f24a84 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:27.774032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8b79f-e09c-402b-ab82-c83097b22388 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:27.782852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 343/686] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/role_assignments?user.id=2c14fb133a0f419390fb0d00c600bdd4&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:27.784017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8b79f-e09c-402b-ab82-c83097b22388 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:27.871525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:27.907851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8b79f-e09c-402b-ab82-c83097b22388 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:27.950260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125a8a9-89b9-4544-810d-725ca6d9c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:27.950894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125a8a9-89b9-4544-810d-725ca6d9c331 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:27.951076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125a8a9-89b9-4544-810d-725ca6d9c331 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:27.951357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125a8a9-89b9-4544-810d-725ca6d9c331 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:27.962047 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125a8a9-89b9-4544-810d-725ca6d9c331 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:27.963812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125a8a9-89b9-4544-810d-725ca6d9c331 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:27.983180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-a125a8a9-89b9-4544-810d-725ca6d9c331 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:27.984741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:27.985887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:27.986936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 344/687] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:27.992402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125a8a9-89b9-4544-810d-725ca6d9c331 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:28.003252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ea6eb-cacc-4b50-a5e1-02b148f24a84 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:28.053594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de2318-7ce5-4a54-adcb-bc85dfcff89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:28.055521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de2318-7ce5-4a54-adcb-bc85dfcff89d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:28.055823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de2318-7ce5-4a54-adcb-bc85dfcff89d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:28.056109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de2318-7ce5-4a54-adcb-bc85dfcff89d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:28.073184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de2318-7ce5-4a54-adcb-bc85dfcff89d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:28.074660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de2318-7ce5-4a54-adcb-bc85dfcff89d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-94de2318-7ce5-4a54-adcb-bc85dfcff89d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:28.092586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:28.093500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:28.094430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:28.094430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:28.094430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:28.094430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:28.101161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 344/688] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:28.102741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de2318-7ce5-4a54-adcb-bc85dfcff89d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:28.120246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125a8a9-89b9-4544-810d-725ca6d9c331 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:28.156124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:28.165774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33516c84-fbef-42d9-8944-4ca567bcbccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:28.166404 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33516c84-fbef-42d9-8944-4ca567bcbccf None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:28.166688 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33516c84-fbef-42d9-8944-4ca567bcbccf None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:28.166966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33516c84-fbef-42d9-8944-4ca567bcbccf None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:28.173601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33516c84-fbef-42d9-8944-4ca567bcbccf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:28.173601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33516c84-fbef-42d9-8944-4ca567bcbccf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:28.173601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33516c84-fbef-42d9-8944-4ca567bcbccf None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:28.194269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 345/689] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Aug 6 01:05:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:28.204794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33516c84-fbef-42d9-8944-4ca567bcbccf None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:28.225418 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de2318-7ce5-4a54-adcb-bc85dfcff89d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:28.268214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a5324-cf99-4140-90f8-eebb70a2b418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:28.269110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a5324-cf99-4140-90f8-eebb70a2b418 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:28.269534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a5324-cf99-4140-90f8-eebb70a2b418 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:28.269957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a5324-cf99-4140-90f8-eebb70a2b418 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:28.270813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a5324-cf99-4140-90f8-eebb70a2b418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:28.271320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a5324-cf99-4140-90f8-eebb70a2b418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:28.273444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a5324-cf99-4140-90f8-eebb70a2b418 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:28.289494 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:28.295714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 345/690] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:28.301134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a5324-cf99-4140-90f8-eebb70a2b418 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:28.339817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33516c84-fbef-42d9-8944-4ca567bcbccf None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:28.373704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df622fb2-8c97-4c86-a580-16f9b3321e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:28.379800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df622fb2-8c97-4c86-a580-16f9b3321e66 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:28.379800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df622fb2-8c97-4c86-a580-16f9b3321e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:28.379800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df622fb2-8c97-4c86-a580-16f9b3321e66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:28.439964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:28.456317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df622fb2-8c97-4c86-a580-16f9b3321e66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:28.458251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df622fb2-8c97-4c86-a580-16f9b3321e66 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:28.496336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a5324-cf99-4140-90f8-eebb70a2b418 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:28.556096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b8382-8072-4591-86a3-9d0cc0eec0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:28.556770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b8382-8072-4591-86a3-9d0cc0eec0de None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:28.557052 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b8382-8072-4591-86a3-9d0cc0eec0de None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:28.557327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b8382-8072-4591-86a3-9d0cc0eec0de None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:28.565258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b8382-8072-4591-86a3-9d0cc0eec0de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:28.566721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b8382-8072-4591-86a3-9d0cc0eec0de None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-2a3b8382-8072-4591-86a3-9d0cc0eec0de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:28.579528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:28.580628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:28.581690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:28.581690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:28.581690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:28.581690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:28.581690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:28.581690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:28.581690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:28.581690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:28.581690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:28.581690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:28.584602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 346/691] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Aug 6 01:05:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:28.590256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b8382-8072-4591-86a3-9d0cc0eec0de None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:28.623957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 346/692] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:28.640818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df622fb2-8c97-4c86-a580-16f9b3321e66 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:28.701389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:28.777907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b8382-8072-4591-86a3-9d0cc0eec0de None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:28.779130 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:28.796417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df622fb2-8c97-4c86-a580-16f9b3321e66 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:28.839445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586f701-a191-4a00-89b4-bfa2b8e70582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:28.841541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586f701-a191-4a00-89b4-bfa2b8e70582 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:28.843157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586f701-a191-4a00-89b4-bfa2b8e70582 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:28.843593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586f701-a191-4a00-89b4-bfa2b8e70582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:28.853084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9261c-b7dc-494c-b9a7-5876968784c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:28.854064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9261c-b7dc-494c-b9a7-5876968784c2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:28.854497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9261c-b7dc-494c-b9a7-5876968784c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:28.854920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9261c-b7dc-494c-b9a7-5876968784c2 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:28.855903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9261c-b7dc-494c-b9a7-5876968784c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:28.856361 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9261c-b7dc-494c-b9a7-5876968784c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:28.858101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9261c-b7dc-494c-b9a7-5876968784c2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:28.873126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586f701-a191-4a00-89b4-bfa2b8e70582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:28.874804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586f701-a191-4a00-89b4-bfa2b8e70582 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-e586f701-a191-4a00-89b4-bfa2b8e70582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:28.906821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:28.907852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:28.908763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:28.908763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:28.908763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:28.908763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:28.917399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 347/693] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:28.917399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd0c53-54d0-4ed7-8b86-56e7a440432f None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:28.917399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd0c53-54d0-4ed7-8b86-56e7a440432f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:28.917399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd0c53-54d0-4ed7-8b86-56e7a440432f None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:28.917972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 348/694] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:28.918429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd0c53-54d0-4ed7-8b86-56e7a440432f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:28.998796 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 347/695] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Aug 6 01:05:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:28.999864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97f961-0363-4e5f-8ff3-df483995191f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.000157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97f961-0363-4e5f-8ff3-df483995191f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.000434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97f961-0363-4e5f-8ff3-df483995191f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.018608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4e97f961-0363-4e5f-8ff3-df483995191f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:29.019825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4e97f961-0363-4e5f-8ff3-df483995191f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:29.031611 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd0c53-54d0-4ed7-8b86-56e7a440432f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:29.068401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38492a-4993-4b19-8dfe-e09bacd13d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:29.069048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38492a-4993-4b19-8dfe-e09bacd13d0d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.069330 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38492a-4993-4b19-8dfe-e09bacd13d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.069608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38492a-4993-4b19-8dfe-e09bacd13d0d None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.070297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38492a-4993-4b19-8dfe-e09bacd13d0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:29.070782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38492a-4993-4b19-8dfe-e09bacd13d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:29.071983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38492a-4993-4b19-8dfe-e09bacd13d0d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:29.079271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:29.085039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 349/696] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:29.085852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38492a-4993-4b19-8dfe-e09bacd13d0d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:29.092177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 348/697] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:29.093246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97f961-0363-4e5f-8ff3-df483995191f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:29.142790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:29.188508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38492a-4993-4b19-8dfe-e09bacd13d0d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:29.200193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97f961-0363-4e5f-8ff3-df483995191f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:29.223409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b60e5-fefd-4cb7-9079-72bd817a8801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:29.223962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b60e5-fefd-4cb7-9079-72bd817a8801 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.224243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b60e5-fefd-4cb7-9079-72bd817a8801 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.224511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b60e5-fefd-4cb7-9079-72bd817a8801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.238193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020847b4-897a-422c-a45e-f5f696ffa877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:29.239116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020847b4-897a-422c-a45e-f5f696ffa877 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.239580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020847b4-897a-422c-a45e-f5f696ffa877 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.240136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020847b4-897a-422c-a45e-f5f696ffa877 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.247810 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020847b4-897a-422c-a45e-f5f696ffa877 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:29.249555 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020847b4-897a-422c-a45e-f5f696ffa877 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-020847b4-897a-422c-a45e-f5f696ffa877 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:29.261767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:29.262703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:29.263628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:29.263628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:29.263628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:29.263628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:29.263628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:29.263628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:29.263628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:29.263628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:29.266156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 349/698] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:29.267195 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020847b4-897a-422c-a45e-f5f696ffa877 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:29.296708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b60e5-fefd-4cb7-9079-72bd817a8801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:29.298415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b60e5-fefd-4cb7-9079-72bd817a8801 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:29.300651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:29.396859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020847b4-897a-422c-a45e-f5f696ffa877 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:29.407922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:29.412308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 350/699] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 01:05:29.414404 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b60e5-fefd-4cb7-9079-72bd817a8801 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:29.442239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c685e-e920-4eff-a329-1e0822bc8ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:29.443005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c685e-e920-4eff-a329-1e0822bc8ffc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.443301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c685e-e920-4eff-a329-1e0822bc8ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.443582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c685e-e920-4eff-a329-1e0822bc8ffc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.444527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c685e-e920-4eff-a329-1e0822bc8ffc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:29.446579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c685e-e920-4eff-a329-1e0822bc8ffc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b38c685e-e920-4eff-a329-1e0822bc8ffc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:29.461350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:29.462342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:29.463204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 350/700] 10.222.0.25 () {56 vars in 1129 bytes} [Sat Aug 6 01:05:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:29.464080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c685e-e920-4eff-a329-1e0822bc8ffc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:29.532299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:29.538687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b60e5-fefd-4cb7-9079-72bd817a8801 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:29.583414 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d16be-ced3-4ba7-9cf2-79197bd2cbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:29.584076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d16be-ced3-4ba7-9cf2-79197bd2cbfd None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.584365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d16be-ced3-4ba7-9cf2-79197bd2cbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.584644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d16be-ced3-4ba7-9cf2-79197bd2cbfd None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.586258 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d16be-ced3-4ba7-9cf2-79197bd2cbfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:29.586574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d16be-ced3-4ba7-9cf2-79197bd2cbfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:29.588760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d16be-ced3-4ba7-9cf2-79197bd2cbfd None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:29.591505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c685e-e920-4eff-a329-1e0822bc8ffc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:29.600812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 351/701] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:29.601635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d16be-ced3-4ba7-9cf2-79197bd2cbfd None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:29.631300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781cf706-dd80-4b60-ab85-377c18c1017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:29.632507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cf706-dd80-4b60-ab85-377c18c1017b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.632793 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cf706-dd80-4b60-ab85-377c18c1017b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.633242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cf706-dd80-4b60-ab85-377c18c1017b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.644038 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cf706-dd80-4b60-ab85-377c18c1017b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:29.645499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781cf706-dd80-4b60-ab85-377c18c1017b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-781cf706-dd80-4b60-ab85-377c18c1017b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:29.660749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:29.661680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:29.662610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:29.662610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:29.662610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:29.662610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:29.662610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 351/702] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:29.663622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc9291-9bc2-4d3e-9326-0fd9fe53f356 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.663622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc9291-9bc2-4d3e-9326-0fd9fe53f356 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.663903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc9291-9bc2-4d3e-9326-0fd9fe53f356 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.665141 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 352/703] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:29.665684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc9291-9bc2-4d3e-9326-0fd9fe53f356 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:29.691989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:29.713056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d16be-ced3-4ba7-9cf2-79197bd2cbfd None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:29.749227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac3d79-3bdb-4562-9f18-12c2d825cffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:29.749851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac3d79-3bdb-4562-9f18-12c2d825cffa None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.750128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac3d79-3bdb-4562-9f18-12c2d825cffa None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.750399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac3d79-3bdb-4562-9f18-12c2d825cffa None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.751067 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac3d79-3bdb-4562-9f18-12c2d825cffa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:29.751370 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac3d79-3bdb-4562-9f18-12c2d825cffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:29.752757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac3d79-3bdb-4562-9f18-12c2d825cffa None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:29.757932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 352/704] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Aug 6 01:05:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:29.763066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac3d79-3bdb-4562-9f18-12c2d825cffa None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:29.774296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc9291-9bc2-4d3e-9326-0fd9fe53f356 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:29.788170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:29.811828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198570cb-11a1-4e1d-91e3-4917245be3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:29.818000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198570cb-11a1-4e1d-91e3-4917245be3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.818000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198570cb-11a1-4e1d-91e3-4917245be3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.818000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198570cb-11a1-4e1d-91e3-4917245be3c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.818000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198570cb-11a1-4e1d-91e3-4917245be3c3 None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=33aa626411f1474595102a4541af901a) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:29.818000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198570cb-11a1-4e1d-91e3-4917245be3c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:29.818000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198570cb-11a1-4e1d-91e3-4917245be3c3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:29.826618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 353/705] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:29 2022] GET /identity/v3/role_assignments?user.id=33aa626411f1474595102a4541af901a&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:29.828185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc249b-6ff6-4971-a318-1daea283109d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.828500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc249b-6ff6-4971-a318-1daea283109d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.828778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc249b-6ff6-4971-a318-1daea283109d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.849845 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-fdfc249b-6ff6-4971-a318-1daea283109d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:29.851104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fdfc249b-6ff6-4971-a318-1daea283109d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:29.885468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac3d79-3bdb-4562-9f18-12c2d825cffa None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:29.934220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 354/706] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:29.935111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269315d-1e25-4294-bbee-0c140c86ec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:29.935239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc249b-6ff6-4971-a318-1daea283109d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:29.936057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269315d-1e25-4294-bbee-0c140c86ec27 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:29.936516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269315d-1e25-4294-bbee-0c140c86ec27 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:29.936961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269315d-1e25-4294-bbee-0c140c86ec27 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:29.938214 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269315d-1e25-4294-bbee-0c140c86ec27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:29.939767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269315d-1e25-4294-bbee-0c140c86ec27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:29.941593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269315d-1e25-4294-bbee-0c140c86ec27 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:29.957277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:29.957668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 353/707] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:29.963352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269315d-1e25-4294-bbee-0c140c86ec27 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:30.072511 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc249b-6ff6-4971-a318-1daea283109d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:30.077257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:30.102703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269315d-1e25-4294-bbee-0c140c86ec27 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:30.111771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32243b-d01a-4a69-ba6f-33728d51f287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:30.112452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32243b-d01a-4a69-ba6f-33728d51f287 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.112732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32243b-d01a-4a69-ba6f-33728d51f287 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.113006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32243b-d01a-4a69-ba6f-33728d51f287 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.113688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32243b-d01a-4a69-ba6f-33728d51f287 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:30.113992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32243b-d01a-4a69-ba6f-33728d51f287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:30.115736 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32243b-d01a-4a69-ba6f-33728d51f287 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:30.122409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 355/708] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Aug 6 01:05:29 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:30.123212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32243b-d01a-4a69-ba6f-33728d51f287 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:30.151761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a1119-a0d1-4d29-b9ab-982d2616103c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:30.152447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a1119-a0d1-4d29-b9ab-982d2616103c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.152726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a1119-a0d1-4d29-b9ab-982d2616103c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.152998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a1119-a0d1-4d29-b9ab-982d2616103c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.165436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a1119-a0d1-4d29-b9ab-982d2616103c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:30.168334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a1119-a0d1-4d29-b9ab-982d2616103c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9f8a1119-a0d1-4d29-b9ab-982d2616103c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:30.185054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:30.186016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:30.186907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:30.186907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:30.186907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:30.186907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:30.186907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:30.186907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:30.186907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:30.186907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:30.194105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 354/709] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:30.204983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a1119-a0d1-4d29-b9ab-982d2616103c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:30.238317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:30.264572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32243b-d01a-4a69-ba6f-33728d51f287 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:30.311694 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52724580-41ba-45cc-bea1-f1f7d6ef1881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:30.312691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52724580-41ba-45cc-bea1-f1f7d6ef1881 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.312791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52724580-41ba-45cc-bea1-f1f7d6ef1881 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.312990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52724580-41ba-45cc-bea1-f1f7d6ef1881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.324401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52724580-41ba-45cc-bea1-f1f7d6ef1881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:30.325793 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52724580-41ba-45cc-bea1-f1f7d6ef1881 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:30.341463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a1119-a0d1-4d29-b9ab-982d2616103c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-52724580-41ba-45cc-bea1-f1f7d6ef1881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:30.344352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:30.345232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:30.346160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:30.346160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:30.346160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:30.346160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:30.346810 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 356/710] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:30.347989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15f244-bd63-4523-a26c-bf502f40eb5e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.348273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15f244-bd63-4523-a26c-bf502f40eb5e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.348546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15f244-bd63-4523-a26c-bf502f40eb5e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.350452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 357/711] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:30.352383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15f244-bd63-4523-a26c-bf502f40eb5e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:30.377261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048ff82-0200-48c6-a434-438c306f35dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:30.377874 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048ff82-0200-48c6-a434-438c306f35dc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.378155 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048ff82-0200-48c6-a434-438c306f35dc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.378431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048ff82-0200-48c6-a434-438c306f35dc None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.379190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048ff82-0200-48c6-a434-438c306f35dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:30.379501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048ff82-0200-48c6-a434-438c306f35dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:30.380961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048ff82-0200-48c6-a434-438c306f35dc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:30.399182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 355/712] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:30.399182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d659017-4024-433e-97ad-d41532a3ae96 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.399182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d659017-4024-433e-97ad-d41532a3ae96 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.399182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d659017-4024-433e-97ad-d41532a3ae96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.403469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:30.415997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5d659017-4024-433e-97ad-d41532a3ae96 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:30.418796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5d659017-4024-433e-97ad-d41532a3ae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:30.486683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15f244-bd63-4523-a26c-bf502f40eb5e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:30.499863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 356/713] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:30.501167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d659017-4024-433e-97ad-d41532a3ae96 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:30.537236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:30.542749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b5d0a-090e-452f-bc2a-b0e6527e5b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:30.543471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b5d0a-090e-452f-bc2a-b0e6527e5b57 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.543759 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b5d0a-090e-452f-bc2a-b0e6527e5b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.544069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b5d0a-090e-452f-bc2a-b0e6527e5b57 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.544911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b5d0a-090e-452f-bc2a-b0e6527e5b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:30.545277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b5d0a-090e-452f-bc2a-b0e6527e5b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:30.546671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b5d0a-090e-452f-bc2a-b0e6527e5b57 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:30.559813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 358/714] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:30.560713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b5d0a-090e-452f-bc2a-b0e6527e5b57 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:30.632341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d659017-4024-433e-97ad-d41532a3ae96 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:30.663577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:30.668830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bcf34-4ae2-465c-903b-028e3048fbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:30.669699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bcf34-4ae2-465c-903b-028e3048fbcc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.670202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bcf34-4ae2-465c-903b-028e3048fbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.670671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bcf34-4ae2-465c-903b-028e3048fbcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.671701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bcf34-4ae2-465c-903b-028e3048fbcc None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=db1b205d165543d0a3a69a2db6af5d1c) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:30.671904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b5d0a-090e-452f-bc2a-b0e6527e5b57 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:30.672733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bcf34-4ae2-465c-903b-028e3048fbcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:30.674365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bcf34-4ae2-465c-903b-028e3048fbcc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:30.687923 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 357/715] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:30 2022] GET /identity/v3/role_assignments?user.id=db1b205d165543d0a3a69a2db6af5d1c&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:30.696796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bcf34-4ae2-465c-903b-028e3048fbcc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:30.709618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0862b-5562-4006-b054-2a638f364467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:30.710380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0862b-5562-4006-b054-2a638f364467 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.710661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0862b-5562-4006-b054-2a638f364467 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.710934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0862b-5562-4006-b054-2a638f364467 None admin] PATH_INFO: `/v3/services/network` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.711626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0862b-5562-4006-b054-2a638f364467 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:30.719818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0862b-5562-4006-b054-2a638f364467 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-5ad0862b-5562-4006-b054-2a638f364467 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:30.737054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:30.738160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:30.739003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 359/716] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:30 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:30.749083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0862b-5562-4006-b054-2a638f364467 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:30.848346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:30.896858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0862b-5562-4006-b054-2a638f364467 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:30.921506 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bcf34-4ae2-465c-903b-028e3048fbcc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:30.932756 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa47425-4d4e-4e50-b1ea-faca19bde9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:30.933436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa47425-4d4e-4e50-b1ea-faca19bde9ec None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.933723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa47425-4d4e-4e50-b1ea-faca19bde9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.933998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa47425-4d4e-4e50-b1ea-faca19bde9ec None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.934793 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa47425-4d4e-4e50-b1ea-faca19bde9ec None admin] RBAC: Adding query filter params (name=network) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:30.935104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa47425-4d4e-4e50-b1ea-faca19bde9ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:30.936628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa47425-4d4e-4e50-b1ea-faca19bde9ec None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:30.941682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 360/717] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:05:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:30.947762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa47425-4d4e-4e50-b1ea-faca19bde9ec None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:30.988490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb5e1e-5ddd-4bc8-9230-a42ceec9fa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:30.989229 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb5e1e-5ddd-4bc8-9230-a42ceec9fa86 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:30.989518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb5e1e-5ddd-4bc8-9230-a42ceec9fa86 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:30.989796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb5e1e-5ddd-4bc8-9230-a42ceec9fa86 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:30.990487 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb5e1e-5ddd-4bc8-9230-a42ceec9fa86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:30.992554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb5e1e-5ddd-4bc8-9230-a42ceec9fa86 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:31.052203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa47425-4d4e-4e50-b1ea-faca19bde9ec None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:31.076031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:31.088715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 358/718] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:05:30 2022] POST /identity/v3/users => generated 254 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:31.094583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5f233-95af-475c-ae05-346614c7bd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:31.095297 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5f233-95af-475c-ae05-346614c7bd0b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:31.095585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5f233-95af-475c-ae05-346614c7bd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:31.095908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5f233-95af-475c-ae05-346614c7bd0b None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:31.096617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5f233-95af-475c-ae05-346614c7bd0b None admin] RBAC: Adding query filter params (type=network) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:31.096991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5f233-95af-475c-ae05-346614c7bd0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:31.098510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5f233-95af-475c-ae05-346614c7bd0b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:31.104958 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 361/719] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:05:30 2022] GET /identity/v3/services?type=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:32.275273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f3c50-1654-481d-92ee-725e51382b0c None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:32.275936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f3c50-1654-481d-92ee-725e51382b0c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:32.275936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f3c50-1654-481d-92ee-725e51382b0c None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:32.277461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 359/720] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:32.282893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa845b0-ea39-4ce6-a502-b5ea15402631 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:32.283298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa845b0-ea39-4ce6-a502-b5ea15402631 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:32.283488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa845b0-ea39-4ce6-a502-b5ea15402631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:32.309137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ffa845b0-ea39-4ce6-a502-b5ea15402631 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:32.310355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ffa845b0-ea39-4ce6-a502-b5ea15402631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:32.402391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 362/721] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 01:05:32.434802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f3c50-1654-481d-92ee-725e51382b0c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:32.507060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:32.543105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87a967-2317-4e5c-813e-63608da6c211 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:32.543299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87a967-2317-4e5c-813e-63608da6c211 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:32.543588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87a967-2317-4e5c-813e-63608da6c211 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:32.545231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 363/722] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:32.550208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df057079-6721-4ad4-b26e-fef17c3667db None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:32.550501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df057079-6721-4ad4-b26e-fef17c3667db None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:32.550779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df057079-6721-4ad4-b26e-fef17c3667db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:32.588519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-df057079-6721-4ad4-b26e-fef17c3667db None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:32.592593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-df057079-6721-4ad4-b26e-fef17c3667db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:32.592748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f3c50-1654-481d-92ee-725e51382b0c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:32.646966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f973b4-b970-4c08-820b-3ee47c0ab09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:32.648300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f973b4-b970-4c08-820b-3ee47c0ab09f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:32.648700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f973b4-b970-4c08-820b-3ee47c0ab09f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:32.648979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f973b4-b970-4c08-820b-3ee47c0ab09f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:32.673711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f973b4-b970-4c08-820b-3ee47c0ab09f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b8f973b4-b970-4c08-820b-3ee47c0ab09f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:32.679412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:32.680473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:32.681394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 360/723] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:32.687240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f973b4-b970-4c08-820b-3ee47c0ab09f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:32.705699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 364/724] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:32.782386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df057079-6721-4ad4-b26e-fef17c3667db None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:32.848282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:32.888996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f973b4-b970-4c08-820b-3ee47c0ab09f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:32.897394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df057079-6721-4ad4-b26e-fef17c3667db None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:32.916299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:32.934575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93b241-62f7-4180-9a54-8b682938acbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:32.935488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93b241-62f7-4180-9a54-8b682938acbb None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:32.935488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93b241-62f7-4180-9a54-8b682938acbb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:32.935743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93b241-62f7-4180-9a54-8b682938acbb None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:32.936448 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93b241-62f7-4180-9a54-8b682938acbb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:32.937940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93b241-62f7-4180-9a54-8b682938acbb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:32.966684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61499c59-2b14-4175-a5fd-46571a8b572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:32.967376 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61499c59-2b14-4175-a5fd-46571a8b572a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:32.967809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61499c59-2b14-4175-a5fd-46571a8b572a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:32.968196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61499c59-2b14-4175-a5fd-46571a8b572a None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:32.968930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61499c59-2b14-4175-a5fd-46571a8b572a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:32.969237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61499c59-2b14-4175-a5fd-46571a8b572a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:32.970599 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61499c59-2b14-4175-a5fd-46571a8b572a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:32.980097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 361/725] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:32.992863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61499c59-2b14-4175-a5fd-46571a8b572a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:33.048954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 365/726] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:05:32 2022] POST /identity/v3/services => generated 251 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:33.183093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb9eb-a24d-4dfe-919e-662ab7def0b0 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:33.183423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb9eb-a24d-4dfe-919e-662ab7def0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:33.183609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eb9eb-a24d-4dfe-919e-662ab7def0b0 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:33.185247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 366/727] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:33.190476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01f64-d235-410c-a949-053ef3e9eae4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:33.190744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01f64-d235-410c-a949-053ef3e9eae4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:33.191024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01f64-d235-410c-a949-053ef3e9eae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:33.227547 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b1d01f64-d235-410c-a949-053ef3e9eae4 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:33.228825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b1d01f64-d235-410c-a949-053ef3e9eae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:33.236748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61499c59-2b14-4175-a5fd-46571a8b572a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:33.287590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce952343-a9be-41a9-aacc-4bfa5a49cba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:33.288924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce952343-a9be-41a9-aacc-4bfa5a49cba6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:33.289097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce952343-a9be-41a9-aacc-4bfa5a49cba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:33.289320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce952343-a9be-41a9-aacc-4bfa5a49cba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:33.300186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce952343-a9be-41a9-aacc-4bfa5a49cba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:33.301957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce952343-a9be-41a9-aacc-4bfa5a49cba6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-ce952343-a9be-41a9-aacc-4bfa5a49cba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:33.317834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:33.318954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:33.320112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:33.320112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:33.320112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:33.320112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:33.322034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 367/729] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:33.322482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 362/729] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:33.328264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d01f64-d235-410c-a949-053ef3e9eae4 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:33.345319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce952343-a9be-41a9-aacc-4bfa5a49cba6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:33.368649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:33.377618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:33.451367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d01f64-d235-410c-a949-053ef3e9eae4 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:33.458630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce952343-a9be-41a9-aacc-4bfa5a49cba6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:33.493038 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c3bcd-6853-4bff-b547-37f190985bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:33.493935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c3bcd-6853-4bff-b547-37f190985bdf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:33.494369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c3bcd-6853-4bff-b547-37f190985bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:33.494797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c3bcd-6853-4bff-b547-37f190985bdf None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:33.495653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c3bcd-6853-4bff-b547-37f190985bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:33.496129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c3bcd-6853-4bff-b547-37f190985bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:33.497661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c3bcd-6853-4bff-b547-37f190985bdf None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:33.503276 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc84cac-76b9-4810-9dcd-613c2df16ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:33.503966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc84cac-76b9-4810-9dcd-613c2df16ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:33.504254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc84cac-76b9-4810-9dcd-613c2df16ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:33.504532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc84cac-76b9-4810-9dcd-613c2df16ea8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:33.510404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 368/730] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:33.511703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d87378-4590-497f-8e70-b1dfc1e88a81 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:33.516400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d87378-4590-497f-8e70-b1dfc1e88a81 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:33.516548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc84cac-76b9-4810-9dcd-613c2df16ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:33.516980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d87378-4590-497f-8e70-b1dfc1e88a81 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:33.518672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 369/731] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:33.519539 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d87378-4590-497f-8e70-b1dfc1e88a81 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-0dc84cac-76b9-4810-9dcd-613c2df16ea8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:33.521286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:33.522399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:33.523503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 363/732] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:33.529933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3f790-82ff-41f3-b6e9-026151da59bf None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:33.530221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3f790-82ff-41f3-b6e9-026151da59bf None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:33.530500 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3f790-82ff-41f3-b6e9-026151da59bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:33.553216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-76b3f790-82ff-41f3-b6e9-026151da59bf None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:33.555243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-76b3f790-82ff-41f3-b6e9-026151da59bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:33.630466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:33.639402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d87378-4590-497f-8e70-b1dfc1e88a81 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:33.658021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 364/733] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 01:05:33.659195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3f790-82ff-41f3-b6e9-026151da59bf None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:33.677579 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:33.679585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7d7df-5c6c-43f4-8032-2c046bfe557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:33.680486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d7df-5c6c-43f4-8032-2c046bfe557f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:33.680971 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d7df-5c6c-43f4-8032-2c046bfe557f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:33.681428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d7df-5c6c-43f4-8032-2c046bfe557f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:33.689724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7d7df-5c6c-43f4-8032-2c046bfe557f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:33.691899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7d7df-5c6c-43f4-8032-2c046bfe557f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-5ea7d7df-5c6c-43f4-8032-2c046bfe557f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:33.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:33.708632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:33.709654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:33.709654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:33.709654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:33.709654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:33.709654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:33.709654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:33.709654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:33.709654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:33.709654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:33.709654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:33.712574 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 370/734] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:33.717922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7d7df-5c6c-43f4-8032-2c046bfe557f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:33.776721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3f790-82ff-41f3-b6e9-026151da59bf None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:33.814391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f892e-7ee3-4142-89a3-361c4aec2ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:33.815023 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f892e-7ee3-4142-89a3-361c4aec2ebd None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:33.815304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f892e-7ee3-4142-89a3-361c4aec2ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:33.815581 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f892e-7ee3-4142-89a3-361c4aec2ebd None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:33.816544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f892e-7ee3-4142-89a3-361c4aec2ebd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:33.816667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f892e-7ee3-4142-89a3-361c4aec2ebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:33.817981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f892e-7ee3-4142-89a3-361c4aec2ebd None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:33.825896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7d7df-5c6c-43f4-8032-2c046bfe557f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:33.831583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 365/735] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:33.832527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f892e-7ee3-4142-89a3-361c4aec2ebd None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:33.863716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348720-a173-4813-90d4-7c56731d586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:33.866516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348720-a173-4813-90d4-7c56731d586c None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:33.866516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348720-a173-4813-90d4-7c56731d586c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:33.866516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348720-a173-4813-90d4-7c56731d586c None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:33.866516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87348720-a173-4813-90d4-7c56731d586c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:33.868025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87348720-a173-4813-90d4-7c56731d586c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:33.911271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:33.936221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f892e-7ee3-4142-89a3-361c4aec2ebd None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:33.961146 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 371/736] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:05:33 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:33.961905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348720-a173-4813-90d4-7c56731d586c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:33.983834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:34.002943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e8c75-d339-4855-a49d-03ffde7f0ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:34.004002 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8c75-d339-4855-a49d-03ffde7f0ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.004292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8c75-d339-4855-a49d-03ffde7f0ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.004567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e8c75-d339-4855-a49d-03ffde7f0ee6 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.005438 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8c75-d339-4855-a49d-03ffde7f0ee6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:34.005950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8c75-d339-4855-a49d-03ffde7f0ee6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:34.008345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e8c75-d339-4855-a49d-03ffde7f0ee6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:34.037481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 366/737] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Aug 6 01:05:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:34.039725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cade98-2ed5-43cf-b303-99682240ee5a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.040137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cade98-2ed5-43cf-b303-99682240ee5a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.040517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cade98-2ed5-43cf-b303-99682240ee5a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.042267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 367/738] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:34.043142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cade98-2ed5-43cf-b303-99682240ee5a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:34.099522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348720-a173-4813-90d4-7c56731d586c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:34.157335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb149a0-e659-4286-9d4d-3d5672aa7d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:34.158273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb149a0-e659-4286-9d4d-3d5672aa7d54 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.158617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb149a0-e659-4286-9d4d-3d5672aa7d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.159376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb149a0-e659-4286-9d4d-3d5672aa7d54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.175913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb149a0-e659-4286-9d4d-3d5672aa7d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:34.177687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb149a0-e659-4286-9d4d-3d5672aa7d54 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:34.184512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cade98-2ed5-43cf-b303-99682240ee5a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-6cb149a0-e659-4286-9d4d-3d5672aa7d54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:34.195502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:34.196605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:34.197520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:34.197520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:34.197520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:34.197520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:34.197520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 372/739] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:34.198686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a0902-cc5f-441a-aef4-02f6a0bb11fe None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.198970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a0902-cc5f-441a-aef4-02f6a0bb11fe None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.199650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a0902-cc5f-441a-aef4-02f6a0bb11fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.220081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-226a0902-cc5f-441a-aef4-02f6a0bb11fe None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:34.221201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-226a0902-cc5f-441a-aef4-02f6a0bb11fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:34.221585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:34.235618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29714156-e364-44b3-8922-bdcaecc312d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:34.236277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29714156-e364-44b3-8922-bdcaecc312d3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.236557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29714156-e364-44b3-8922-bdcaecc312d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.236831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29714156-e364-44b3-8922-bdcaecc312d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.307945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:34.314247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29714156-e364-44b3-8922-bdcaecc312d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:34.315940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29714156-e364-44b3-8922-bdcaecc312d3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:34.320151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 373/740] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:34.321206 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a0902-cc5f-441a-aef4-02f6a0bb11fe None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:34.441531 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a0902-cc5f-441a-aef4-02f6a0bb11fe None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:34.446782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 368/741] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:05:34.447658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29714156-e364-44b3-8922-bdcaecc312d3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:34.478011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:34.481265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead89a53-ba29-480d-8c10-3b1fd9235480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:34.481989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead89a53-ba29-480d-8c10-3b1fd9235480 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.482277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead89a53-ba29-480d-8c10-3b1fd9235480 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.482675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead89a53-ba29-480d-8c10-3b1fd9235480 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.483383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead89a53-ba29-480d-8c10-3b1fd9235480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:34.483798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead89a53-ba29-480d-8c10-3b1fd9235480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:34.489506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead89a53-ba29-480d-8c10-3b1fd9235480 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:34.498663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 374/742] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:34.499660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead89a53-ba29-480d-8c10-3b1fd9235480 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:34.553426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29714156-e364-44b3-8922-bdcaecc312d3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:34.568145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:34.592576 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a89d6-b21a-40d7-8e56-a04a220d8aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:34.593443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a89d6-b21a-40d7-8e56-a04a220d8aac None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.593872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a89d6-b21a-40d7-8e56-a04a220d8aac None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.594294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a89d6-b21a-40d7-8e56-a04a220d8aac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.605079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a89d6-b21a-40d7-8e56-a04a220d8aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:34.609174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead89a53-ba29-480d-8c10-3b1fd9235480 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-525a89d6-b21a-40d7-8e56-a04a220d8aac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:34.610592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:34.611726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:34.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:34.615103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 369/743] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:34.616101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a89d6-b21a-40d7-8e56-a04a220d8aac None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:34.653883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c0a7f-2a12-43b5-b277-c9d8e8c365ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:34.654536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c0a7f-2a12-43b5-b277-c9d8e8c365ee None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.654818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c0a7f-2a12-43b5-b277-c9d8e8c365ee None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.655092 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c0a7f-2a12-43b5-b277-c9d8e8c365ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.669258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c0a7f-2a12-43b5-b277-c9d8e8c365ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:34.671101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c0a7f-2a12-43b5-b277-c9d8e8c365ee None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ee5c0a7f-2a12-43b5-b277-c9d8e8c365ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:34.690808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:34.691744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:34.692687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:34.692687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:34.692687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:34.692687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:34.692687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 375/744] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:34.693424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c0a7f-2a12-43b5-b277-c9d8e8c365ee None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:34.714681 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:34.732394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a89d6-b21a-40d7-8e56-a04a220d8aac None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:34.769110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e755e-9b50-42fb-9d26-c280c9956724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:34.770125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e755e-9b50-42fb-9d26-c280c9956724 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.770617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e755e-9b50-42fb-9d26-c280c9956724 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.771100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e755e-9b50-42fb-9d26-c280c9956724 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.778697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e755e-9b50-42fb-9d26-c280c9956724 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:34.780396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e755e-9b50-42fb-9d26-c280c9956724 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3d3e755e-9b50-42fb-9d26-c280c9956724 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:34.793305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:34.794340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:34.795319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:34.795319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:34.795319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:34.795319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:34.795319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:34.795319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:34.795319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:34.795319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:34.795319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:34.795319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:34.797948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 370/745] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Aug 6 01:05:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:34.798953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e755e-9b50-42fb-9d26-c280c9956724 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:34.823599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c0a7f-2a12-43b5-b277-c9d8e8c365ee None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:34.836157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:34.870154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8080011-1461-44cf-8fbe-c8158b557d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:34.870782 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8080011-1461-44cf-8fbe-c8158b557d47 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.871064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8080011-1461-44cf-8fbe-c8158b557d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.871342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8080011-1461-44cf-8fbe-c8158b557d47 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.872346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8080011-1461-44cf-8fbe-c8158b557d47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:34.872663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8080011-1461-44cf-8fbe-c8158b557d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:34.874763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8080011-1461-44cf-8fbe-c8158b557d47 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:34.887853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 376/746] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:34.889033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8080011-1461-44cf-8fbe-c8158b557d47 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:34.928101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e755e-9b50-42fb-9d26-c280c9956724 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:34.975417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d0d66-c0cd-4bba-abac-4ce2a68b3dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:34.976600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d0d66-c0cd-4bba-abac-4ce2a68b3dde None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:34.976885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d0d66-c0cd-4bba-abac-4ce2a68b3dde None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:34.977162 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d0d66-c0cd-4bba-abac-4ce2a68b3dde None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:34.978133 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d0d66-c0cd-4bba-abac-4ce2a68b3dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:34.978443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d0d66-c0cd-4bba-abac-4ce2a68b3dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:34.980062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d0d66-c0cd-4bba-abac-4ce2a68b3dde None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:34.997713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 371/747] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:34.998725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d0d66-c0cd-4bba-abac-4ce2a68b3dde None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:35.001049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8080011-1461-44cf-8fbe-c8158b557d47 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:35.011297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:35.037231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522d9bf-9ca0-41e6-bb55-26fa6fd3be4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:35.038742 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522d9bf-9ca0-41e6-bb55-26fa6fd3be4c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.039227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522d9bf-9ca0-41e6-bb55-26fa6fd3be4c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.039719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522d9bf-9ca0-41e6-bb55-26fa6fd3be4c None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.040754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522d9bf-9ca0-41e6-bb55-26fa6fd3be4c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:35.041264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522d9bf-9ca0-41e6-bb55-26fa6fd3be4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:35.042923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522d9bf-9ca0-41e6-bb55-26fa6fd3be4c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:35.062445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 377/748] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Aug 6 01:05:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:35.063596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522d9bf-9ca0-41e6-bb55-26fa6fd3be4c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:35.093775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:35.133409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d0d66-c0cd-4bba-abac-4ce2a68b3dde None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:35.172304 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522d9bf-9ca0-41e6-bb55-26fa6fd3be4c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:35.180497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e4c7f-accc-4670-a8e4-be1b2158bc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:35.181147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e4c7f-accc-4670-a8e4-be1b2158bc0a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.181430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e4c7f-accc-4670-a8e4-be1b2158bc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.181742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e4c7f-accc-4670-a8e4-be1b2158bc0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.194946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e4c7f-accc-4670-a8e4-be1b2158bc0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:35.196844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e4c7f-accc-4670-a8e4-be1b2158bc0a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:35.211136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f9004-9f47-43e2-88e9-09c4091c3a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:35.212185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f9004-9f47-43e2-88e9-09c4091c3a06 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.212676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f9004-9f47-43e2-88e9-09c4091c3a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.213166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f9004-9f47-43e2-88e9-09c4091c3a06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-c03e4c7f-accc-4670-a8e4-be1b2158bc0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:35.215034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:35.215952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:35.216869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:35.216869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:35.216869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:35.216869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:35.219921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 372/749] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:35.221090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e4c7f-accc-4670-a8e4-be1b2158bc0a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:35.228705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f9004-9f47-43e2-88e9-09c4091c3a06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:35.229529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f9004-9f47-43e2-88e9-09c4091c3a06 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-499f9004-9f47-43e2-88e9-09c4091c3a06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:35.242370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:35.243359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:35.244693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:35.244693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:35.244693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:35.244693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:35.244693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:35.244693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:35.244693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:35.244693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:35.244693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 378/750] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:35.244693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f9004-9f47-43e2-88e9-09c4091c3a06 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:35.320295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:35.338387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:35.350659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f9004-9f47-43e2-88e9-09c4091c3a06 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:35.360564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e4c7f-accc-4670-a8e4-be1b2158bc0a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:35.387459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9715f6-e9b1-4d9e-ab3d-4e200f1d7b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:35.388121 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9715f6-e9b1-4d9e-ab3d-4e200f1d7b77 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.388409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9715f6-e9b1-4d9e-ab3d-4e200f1d7b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.388682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9715f6-e9b1-4d9e-ab3d-4e200f1d7b77 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.389468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9715f6-e9b1-4d9e-ab3d-4e200f1d7b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:35.389761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9715f6-e9b1-4d9e-ab3d-4e200f1d7b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:35.391143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9715f6-e9b1-4d9e-ab3d-4e200f1d7b77 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:35.397224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c02369-c042-425c-800a-71c60eb17d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:35.398420 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c02369-c042-425c-800a-71c60eb17d3e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.398702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c02369-c042-425c-800a-71c60eb17d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.398977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c02369-c042-425c-800a-71c60eb17d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.403430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 379/751] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:35.404307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9715f6-e9b1-4d9e-ab3d-4e200f1d7b77 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:35.415121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c02369-c042-425c-800a-71c60eb17d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:35.416989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c02369-c042-425c-800a-71c60eb17d3e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-d1c02369-c042-425c-800a-71c60eb17d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:35.431813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:35.432904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:35.434097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:35.434097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:35.434097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:35.434097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:35.437772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 373/752] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:35.438822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c019a64-1f7a-4055-a0aa-4dda4248cc67 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.439107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c019a64-1f7a-4055-a0aa-4dda4248cc67 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.439379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c019a64-1f7a-4055-a0aa-4dda4248cc67 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.443829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 374/753] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:35.443829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c019a64-1f7a-4055-a0aa-4dda4248cc67 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:35.521417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9715f6-e9b1-4d9e-ab3d-4e200f1d7b77 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:35.546261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c019a64-1f7a-4055-a0aa-4dda4248cc67 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:35.558973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:35.559129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b577-b3c2-4479-a0d2-4585fb22ea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:35.560076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b577-b3c2-4479-a0d2-4585fb22ea65 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.560357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b577-b3c2-4479-a0d2-4585fb22ea65 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.560630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b577-b3c2-4479-a0d2-4585fb22ea65 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.561381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672b577-b3c2-4479-a0d2-4585fb22ea65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:35.561809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672b577-b3c2-4479-a0d2-4585fb22ea65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:35.563248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672b577-b3c2-4479-a0d2-4585fb22ea65 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:35.582608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 380/754] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:35.582608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b577-b3c2-4479-a0d2-4585fb22ea65 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:35.592138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2264d-4a16-4a68-af90-141ab0496587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:35.592785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2264d-4a16-4a68-af90-141ab0496587 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.593064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2264d-4a16-4a68-af90-141ab0496587 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.593336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2264d-4a16-4a68-af90-141ab0496587 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.593632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:35.601539 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2264d-4a16-4a68-af90-141ab0496587 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:35.602993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2264d-4a16-4a68-af90-141ab0496587 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-53a2264d-4a16-4a68-af90-141ab0496587 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:35.616001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:35.616987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 375/755] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Aug 6 01:05:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:35.618397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a8089-a15c-48ed-a322-ac84003074ee None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.619116 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a8089-a15c-48ed-a322-ac84003074ee None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.619116 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a8089-a15c-48ed-a322-ac84003074ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.638564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-288a8089-a15c-48ed-a322-ac84003074ee None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:35.639758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-288a8089-a15c-48ed-a322-ac84003074ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:35.694920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b577-b3c2-4479-a0d2-4585fb22ea65 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:35.711482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 376/756] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:35.712587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a8089-a15c-48ed-a322-ac84003074ee None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:35.744168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cee1d-8be1-4eb4-8d8d-43be8b6ecb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:35.744892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cee1d-8be1-4eb4-8d8d-43be8b6ecb09 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.745181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cee1d-8be1-4eb4-8d8d-43be8b6ecb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.745456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cee1d-8be1-4eb4-8d8d-43be8b6ecb09 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.746147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380cee1d-8be1-4eb4-8d8d-43be8b6ecb09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:35.746461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380cee1d-8be1-4eb4-8d8d-43be8b6ecb09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:35.747869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380cee1d-8be1-4eb4-8d8d-43be8b6ecb09 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:35.760951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 381/757] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:35.761583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cee1d-8be1-4eb4-8d8d-43be8b6ecb09 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:35.828677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a8089-a15c-48ed-a322-ac84003074ee None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:35.828888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:35.830463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:35.862916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cee1d-8be1-4eb4-8d8d-43be8b6ecb09 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:35.863530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be55c5-6c30-4c43-b339-3ccdfff6a937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:35.864321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be55c5-6c30-4c43-b339-3ccdfff6a937 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.864683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be55c5-6c30-4c43-b339-3ccdfff6a937 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.865035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be55c5-6c30-4c43-b339-3ccdfff6a937 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.865921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be55c5-6c30-4c43-b339-3ccdfff6a937 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=33aa626411f1474595102a4541af901a) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:35.866418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be55c5-6c30-4c43-b339-3ccdfff6a937 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:35.867899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be55c5-6c30-4c43-b339-3ccdfff6a937 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:35.883825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 377/758] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:35 2022] GET /identity/v3/role_assignments?user.id=33aa626411f1474595102a4541af901a&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:35.883825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be55c5-6c30-4c43-b339-3ccdfff6a937 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:35.922094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43964ced-460b-4aab-a36d-a323ef21455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:35.923722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43964ced-460b-4aab-a36d-a323ef21455c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:35.924064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43964ced-460b-4aab-a36d-a323ef21455c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:35.924349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43964ced-460b-4aab-a36d-a323ef21455c None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:35.925177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43964ced-460b-4aab-a36d-a323ef21455c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:35.925492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43964ced-460b-4aab-a36d-a323ef21455c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:35.927015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43964ced-460b-4aab-a36d-a323ef21455c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:35.955976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 382/759] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Aug 6 01:05:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:35.956993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43964ced-460b-4aab-a36d-a323ef21455c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:36.008644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be55c5-6c30-4c43-b339-3ccdfff6a937 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:36.045342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69297c4-c8a9-4f17-ad06-6e0ae12052f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:36.046128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69297c4-c8a9-4f17-ad06-6e0ae12052f5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.046486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69297c4-c8a9-4f17-ad06-6e0ae12052f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.046830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69297c4-c8a9-4f17-ad06-6e0ae12052f5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.047538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69297c4-c8a9-4f17-ad06-6e0ae12052f5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:36.049020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69297c4-c8a9-4f17-ad06-6e0ae12052f5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f69297c4-c8a9-4f17-ad06-6e0ae12052f5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:36.061394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:36.062625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:05:36.063669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:36.064908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 378/760] 10.222.0.25 () {56 vars in 1129 bytes} [Sat Aug 6 01:05:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:36.065660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69297c4-c8a9-4f17-ad06-6e0ae12052f5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:36.074110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43964ced-460b-4aab-a36d-a323ef21455c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:36.102297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:36.108991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:36.116704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f15d43-167a-4694-8100-edd8fa70940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:36.116704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f15d43-167a-4694-8100-edd8fa70940f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.116704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f15d43-167a-4694-8100-edd8fa70940f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.116704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f15d43-167a-4694-8100-edd8fa70940f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.122668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f15d43-167a-4694-8100-edd8fa70940f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:36.124343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f15d43-167a-4694-8100-edd8fa70940f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-00f15d43-167a-4694-8100-edd8fa70940f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:36.138452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:36.144382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:36.145295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:36.145295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:36.145295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:36.145295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:36.145295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:36.145295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:36.145295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:36.145295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:36.145295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 383/761] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:36.145295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f15d43-167a-4694-8100-edd8fa70940f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:36.185596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69297c4-c8a9-4f17-ad06-6e0ae12052f5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:36.222239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c3593-1b81-4af3-b940-c253534ad477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:36.223020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c3593-1b81-4af3-b940-c253534ad477 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.223402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c3593-1b81-4af3-b940-c253534ad477 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.223797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c3593-1b81-4af3-b940-c253534ad477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.249111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f15d43-167a-4694-8100-edd8fa70940f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:36.284667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dfd25-3251-451a-bef0-a3facc4f1c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:36.285299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dfd25-3251-451a-bef0-a3facc4f1c7a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.285582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dfd25-3251-451a-bef0-a3facc4f1c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.285857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dfd25-3251-451a-bef0-a3facc4f1c7a None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.286532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463dfd25-3251-451a-bef0-a3facc4f1c7a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:36.286838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463dfd25-3251-451a-bef0-a3facc4f1c7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:36.290076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463dfd25-3251-451a-bef0-a3facc4f1c7a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:36.300465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 384/762] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Aug 6 01:05:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:36.300465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dfd25-3251-451a-bef0-a3facc4f1c7a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:36.311559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c3593-1b81-4af3-b940-c253534ad477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:36.313036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c3593-1b81-4af3-b940-c253534ad477 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:36.344303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:36.355530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:36.407041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dfd25-3251-451a-bef0-a3facc4f1c7a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:36.415671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 379/763] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:05:36.416966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968860df-e05b-4544-8c18-9e35880cc9f2 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.417259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968860df-e05b-4544-8c18-9e35880cc9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.417540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968860df-e05b-4544-8c18-9e35880cc9f2 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.422718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 380/764] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:36.422718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968860df-e05b-4544-8c18-9e35880cc9f2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:36.450551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912480e5-3a09-47f0-be4f-59c7eba4ee4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:36.451449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912480e5-3a09-47f0-be4f-59c7eba4ee4c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.451917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912480e5-3a09-47f0-be4f-59c7eba4ee4c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.452360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912480e5-3a09-47f0-be4f-59c7eba4ee4c None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.453513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912480e5-3a09-47f0-be4f-59c7eba4ee4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:36.453975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912480e5-3a09-47f0-be4f-59c7eba4ee4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:36.455571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912480e5-3a09-47f0-be4f-59c7eba4ee4c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:36.468490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 385/765] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:36.469532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912480e5-3a09-47f0-be4f-59c7eba4ee4c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:36.531647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968860df-e05b-4544-8c18-9e35880cc9f2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:36.559962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:36.565477 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92e733-d3c6-441f-b07c-94ae8e73de24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:36.566177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92e733-d3c6-441f-b07c-94ae8e73de24 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.566501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92e733-d3c6-441f-b07c-94ae8e73de24 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.572471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92e733-d3c6-441f-b07c-94ae8e73de24 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.572471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92e733-d3c6-441f-b07c-94ae8e73de24 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:36.572471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92e733-d3c6-441f-b07c-94ae8e73de24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:36.572471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92e733-d3c6-441f-b07c-94ae8e73de24 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:36.575149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 381/766] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Aug 6 01:05:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:36.576368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f238e8-6a3e-4d9f-97dc-ebf128b9fe15 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.576660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f238e8-6a3e-4d9f-97dc-ebf128b9fe15 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.576941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f238e8-6a3e-4d9f-97dc-ebf128b9fe15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.589806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912480e5-3a09-47f0-be4f-59c7eba4ee4c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:36.598438 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c3f238e8-6a3e-4d9f-97dc-ebf128b9fe15 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:36.599632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c3f238e8-6a3e-4d9f-97dc-ebf128b9fe15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:36.603566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:36.629388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa5693-1c4b-4a9f-bd28-26d8e5023f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:36.630207 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa5693-1c4b-4a9f-bd28-26d8e5023f0a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.630965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa5693-1c4b-4a9f-bd28-26d8e5023f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.631709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa5693-1c4b-4a9f-bd28-26d8e5023f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.641943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa5693-1c4b-4a9f-bd28-26d8e5023f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:36.643393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa5693-1c4b-4a9f-bd28-26d8e5023f0a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4aaa5693-1c4b-4a9f-bd28-26d8e5023f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:36.657747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:36.658647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:36.659690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:36.659690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:36.659690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:36.659690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:36.663990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 386/767] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:36.664738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa5693-1c4b-4a9f-bd28-26d8e5023f0a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:36.683799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 382/768] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:36.683799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f238e8-6a3e-4d9f-97dc-ebf128b9fe15 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:36.763243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa5693-1c4b-4a9f-bd28-26d8e5023f0a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:36.793074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f238e8-6a3e-4d9f-97dc-ebf128b9fe15 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:36.802506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137a703-c32c-436c-b48a-92b69bcb51a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:36.803441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137a703-c32c-436c-b48a-92b69bcb51a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.803880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137a703-c32c-436c-b48a-92b69bcb51a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.804165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137a703-c32c-436c-b48a-92b69bcb51a9 None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/db1b205d165543d0a3a69a2db6af5d1c/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.806091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137a703-c32c-436c-b48a-92b69bcb51a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=db1b205d165543d0a3a69a2db6af5d1c, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:36.807517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137a703-c32c-436c-b48a-92b69bcb51a9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:36.817114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:36.827104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59b468-071d-4f21-8da0-26ea995d801b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:36.829731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59b468-071d-4f21-8da0-26ea995d801b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:36.829731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59b468-071d-4f21-8da0-26ea995d801b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:36.829731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59b468-071d-4f21-8da0-26ea995d801b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:36.829731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59b468-071d-4f21-8da0-26ea995d801b None admin] RBAC: Adding query filter params (interface=public, service_id=a25348afe20b4a90a398d9170fa6fd3b, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:36.829731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59b468-071d-4f21-8da0-26ea995d801b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:36.830867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59b468-071d-4f21-8da0-26ea995d801b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:36.838537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 383/769] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Aug 6 01:05:36 2022] GET /identity/v3/endpoints?service_id=a25348afe20b4a90a398d9170fa6fd3b&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:36.839300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59b468-071d-4f21-8da0-26ea995d801b None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:36.891811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 387/770] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:05:36 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/db1b205d165543d0a3a69a2db6af5d1c/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:05:36.891811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137a703-c32c-436c-b48a-92b69bcb51a9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:36.921560 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:36.967835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59b468-071d-4f21-8da0-26ea995d801b None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:37.010233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47f20-c70f-48ce-9102-e52de8538b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:37.010984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47f20-c70f-48ce-9102-e52de8538b99 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:37.011368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47f20-c70f-48ce-9102-e52de8538b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:37.011712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47f20-c70f-48ce-9102-e52de8538b99 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:37.012583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe47f20-c70f-48ce-9102-e52de8538b99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:37.012958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe47f20-c70f-48ce-9102-e52de8538b99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:37.014510 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe47f20-c70f-48ce-9102-e52de8538b99 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:37.030275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137a703-c32c-436c-b48a-92b69bcb51a9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:37.032353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 384/771] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:37.033397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47f20-c70f-48ce-9102-e52de8538b99 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:37.075183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634dcfd-186a-48f4-9fda-2b9383bda6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:37.076578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634dcfd-186a-48f4-9fda-2b9383bda6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:37.076578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634dcfd-186a-48f4-9fda-2b9383bda6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:37.076578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634dcfd-186a-48f4-9fda-2b9383bda6e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:37.077041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634dcfd-186a-48f4-9fda-2b9383bda6e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:37.078674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634dcfd-186a-48f4-9fda-2b9383bda6e5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7634dcfd-186a-48f4-9fda-2b9383bda6e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:37.089666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:37.090735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:37.091616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 388/772] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:36 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:37.092517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634dcfd-186a-48f4-9fda-2b9383bda6e5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:37.097754 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:37.155586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47f20-c70f-48ce-9102-e52de8538b99 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:37.199637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291141b1-7012-4783-9e62-b2e628fd2123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:37.201545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291141b1-7012-4783-9e62-b2e628fd2123 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:37.206246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291141b1-7012-4783-9e62-b2e628fd2123 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:37.206246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291141b1-7012-4783-9e62-b2e628fd2123 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:37.206246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291141b1-7012-4783-9e62-b2e628fd2123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:37.206246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291141b1-7012-4783-9e62-b2e628fd2123 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:37.206608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:37.214161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 385/773] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Aug 6 01:05:37 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:37.215044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291141b1-7012-4783-9e62-b2e628fd2123 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:37.223045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634dcfd-186a-48f4-9fda-2b9383bda6e5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:37.268906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97354197-0030-4f8c-9324-7ddf15777021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:37.269683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97354197-0030-4f8c-9324-7ddf15777021 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:37.270022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97354197-0030-4f8c-9324-7ddf15777021 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:37.270354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97354197-0030-4f8c-9324-7ddf15777021 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:37.279006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97354197-0030-4f8c-9324-7ddf15777021 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:37.280829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97354197-0030-4f8c-9324-7ddf15777021 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-97354197-0030-4f8c-9324-7ddf15777021 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:37.294925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:37.296102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:37.297223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:37.297223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:37.297223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:37.297223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:37.297223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:37.297223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:37.297223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:37.297223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:37.304038 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 389/774] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:37.305761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97354197-0030-4f8c-9324-7ddf15777021 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:37.418205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291141b1-7012-4783-9e62-b2e628fd2123 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:37.435229 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:37.444089 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97354197-0030-4f8c-9324-7ddf15777021 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:37.468146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d473-3800-4d8e-99a9-d3f9199f0de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:37.468752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d473-3800-4d8e-99a9-d3f9199f0de1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:37.469242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d473-3800-4d8e-99a9-d3f9199f0de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:37.470014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d473-3800-4d8e-99a9-d3f9199f0de1 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:37.470948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78d473-3800-4d8e-99a9-d3f9199f0de1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:37.471976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78d473-3800-4d8e-99a9-d3f9199f0de1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:37.473613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78d473-3800-4d8e-99a9-d3f9199f0de1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:37.480902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 386/775] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:05:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:37.486416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d473-3800-4d8e-99a9-d3f9199f0de1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:37.500543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad4b1f-95f9-49c9-96ce-17e72533cee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:37.501336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad4b1f-95f9-49c9-96ce-17e72533cee2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:37.501619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad4b1f-95f9-49c9-96ce-17e72533cee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:37.501901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad4b1f-95f9-49c9-96ce-17e72533cee2 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:37.502776 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad4b1f-95f9-49c9-96ce-17e72533cee2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:37.503092 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad4b1f-95f9-49c9-96ce-17e72533cee2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:37.504608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad4b1f-95f9-49c9-96ce-17e72533cee2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:37.520548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 390/776] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:37.527429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad4b1f-95f9-49c9-96ce-17e72533cee2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:37.572219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:37.634163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad4b1f-95f9-49c9-96ce-17e72533cee2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:37.654079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d473-3800-4d8e-99a9-d3f9199f0de1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:37.697753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43ca5-88cd-46a3-a173-86a2042d1fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:37.698444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43ca5-88cd-46a3-a173-86a2042d1fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:37.698734 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43ca5-88cd-46a3-a173-86a2042d1fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:37.699015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43ca5-88cd-46a3-a173-86a2042d1fd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:37.699838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f43ca5-88cd-46a3-a173-86a2042d1fd8 None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=6d62ec1463c14665b94e9b2d85e48e14) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:37.700159 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f43ca5-88cd-46a3-a173-86a2042d1fd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:37.701611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f43ca5-88cd-46a3-a173-86a2042d1fd8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:37.713729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 391/777] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:37 2022] GET /identity/v3/role_assignments?user.id=6d62ec1463c14665b94e9b2d85e48e14&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:37.716885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a46f0-cedc-4f67-9bdb-0de259e94665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:37.717528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a46f0-cedc-4f67-9bdb-0de259e94665 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:37.717812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a46f0-cedc-4f67-9bdb-0de259e94665 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:37.718090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a46f0-cedc-4f67-9bdb-0de259e94665 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:37.718923 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a46f0-cedc-4f67-9bdb-0de259e94665 None admin] RBAC: Adding query filter params (type=network) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:37.719181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a46f0-cedc-4f67-9bdb-0de259e94665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:37.720630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a46f0-cedc-4f67-9bdb-0de259e94665 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:37.726762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 387/778] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:05:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:37.732493 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43ca5-88cd-46a3-a173-86a2042d1fd8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:37.849087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:37.888564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43ca5-88cd-46a3-a173-86a2042d1fd8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:37.960996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321fd7f-e025-4a34-bfc5-d306e2755826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:37.961632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321fd7f-e025-4a34-bfc5-d306e2755826 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:37.961919 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321fd7f-e025-4a34-bfc5-d306e2755826 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:37.962200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321fd7f-e025-4a34-bfc5-d306e2755826 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:37.962966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321fd7f-e025-4a34-bfc5-d306e2755826 None admin] RBAC: Adding query filter params (interface=public, service_id=219a86516aef4b53877383a29d95b3e0, region_id=RegionOne) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:37.963286 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321fd7f-e025-4a34-bfc5-d306e2755826 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:37.964713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321fd7f-e025-4a34-bfc5-d306e2755826 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:37.978503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 392/779] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Aug 6 01:05:37 2022] GET /identity/v3/endpoints?service_id=219a86516aef4b53877383a29d95b3e0&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:37.984020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a46f0-cedc-4f67-9bdb-0de259e94665 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:38.029806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5ab43-be50-433b-8062-edf2db9713f9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:38.030131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5ab43-be50-433b-8062-edf2db9713f9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:38.030378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5ab43-be50-433b-8062-edf2db9713f9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:38.031740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 393/780] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:05:38.037980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b516bb-e0cd-4bfe-b90f-7098829b6b5c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:38.038817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b516bb-e0cd-4bfe-b90f-7098829b6b5c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:38.039095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b516bb-e0cd-4bfe-b90f-7098829b6b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:38.065964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d4b516bb-e0cd-4bfe-b90f-7098829b6b5c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:38.067173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d4b516bb-e0cd-4bfe-b90f-7098829b6b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:38.069423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:38.126967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a46f0-cedc-4f67-9bdb-0de259e94665 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:38.160873 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bedaef-54cf-43d8-9f26-b69ece4301db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:38.161494 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bedaef-54cf-43d8-9f26-b69ece4301db None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:38.161775 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bedaef-54cf-43d8-9f26-b69ece4301db None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:38.162050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bedaef-54cf-43d8-9f26-b69ece4301db None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:38.162704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bedaef-54cf-43d8-9f26-b69ece4301db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:38.164646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bedaef-54cf-43d8-9f26-b69ece4301db None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:38.170125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 388/781] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Aug 6 01:05:37 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:38.179884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 394/782] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:05:38.197425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bedaef-54cf-43d8-9f26-b69ece4301db None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:38.305181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bedaef-54cf-43d8-9f26-b69ece4301db None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:38.307519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:38.349689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310e208-8816-47ce-9298-74fb4593e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:38.350366 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310e208-8816-47ce-9298-74fb4593e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:38.351223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310e208-8816-47ce-9298-74fb4593e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:38.351515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310e208-8816-47ce-9298-74fb4593e6b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:38.369156 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310e208-8816-47ce-9298-74fb4593e6b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-9310e208-8816-47ce-9298-74fb4593e6b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:38.374144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:38.375097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:38.375987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 389/783] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:38.384936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b516bb-e0cd-4bfe-b90f-7098829b6b5c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:38.472242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:38.511573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b516bb-e0cd-4bfe-b90f-7098829b6b5c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:38.546078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ff157-914e-4e0d-9c09-ba6a3fa54135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:38.546703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ff157-914e-4e0d-9c09-ba6a3fa54135 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:38.546982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ff157-914e-4e0d-9c09-ba6a3fa54135 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:38.547254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ff157-914e-4e0d-9c09-ba6a3fa54135 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:38.547995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ff157-914e-4e0d-9c09-ba6a3fa54135 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:38.548302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ff157-914e-4e0d-9c09-ba6a3fa54135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:38.549734 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ff157-914e-4e0d-9c09-ba6a3fa54135 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:38.563208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 395/784] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:38.565083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310e208-8816-47ce-9298-74fb4593e6b4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:38.705044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310e208-8816-47ce-9298-74fb4593e6b4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:38.751308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74e390-c1a2-4889-a8ff-85a66b7fb5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:38.751978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74e390-c1a2-4889-a8ff-85a66b7fb5ea None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:38.752266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74e390-c1a2-4889-a8ff-85a66b7fb5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:38.752541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74e390-c1a2-4889-a8ff-85a66b7fb5ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:38.763854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74e390-c1a2-4889-a8ff-85a66b7fb5ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:38.765305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74e390-c1a2-4889-a8ff-85a66b7fb5ea None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3e74e390-c1a2-4889-a8ff-85a66b7fb5ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:38.786558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:38.787498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:38.788473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:38.788473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:38.788473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:38.788473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:38.792098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 390/785] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:38.793487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ff157-914e-4e0d-9c09-ba6a3fa54135 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:38.978993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ff157-914e-4e0d-9c09-ba6a3fa54135 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:39.013050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:39.022364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921f326-311e-4f63-b91d-80a67b0e6f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:39.023024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f326-311e-4f63-b91d-80a67b0e6f06 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:39.023306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f326-311e-4f63-b91d-80a67b0e6f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:39.023579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f326-311e-4f63-b91d-80a67b0e6f06 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:39.024296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921f326-311e-4f63-b91d-80a67b0e6f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:39.024599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921f326-311e-4f63-b91d-80a67b0e6f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:39.025959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921f326-311e-4f63-b91d-80a67b0e6f06 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:39.043017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 396/786] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:39.048789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74e390-c1a2-4889-a8ff-85a66b7fb5ea None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:39.082096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:39.184028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74e390-c1a2-4889-a8ff-85a66b7fb5ea None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:39.226671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762b9f7-1c9b-48a2-9899-ab7f1997e511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:39.227577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b9f7-1c9b-48a2-9899-ab7f1997e511 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:39.228035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b9f7-1c9b-48a2-9899-ab7f1997e511 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:39.228458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b9f7-1c9b-48a2-9899-ab7f1997e511 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:39.236287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762b9f7-1c9b-48a2-9899-ab7f1997e511 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:39.237904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762b9f7-1c9b-48a2-9899-ab7f1997e511 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8762b9f7-1c9b-48a2-9899-ab7f1997e511 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:39.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:39.256696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:39.258443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:39.258443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:39.258443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:39.258443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:39.258443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:39.258443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:39.258443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:39.258443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:39.258443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:39.258443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:39.260320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 391/787] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:39.266313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921f326-311e-4f63-b91d-80a67b0e6f06 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:39.404575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921f326-311e-4f63-b91d-80a67b0e6f06 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:39.457443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e37c67-439e-44c4-a9ea-3436286bcdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:39.458082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e37c67-439e-44c4-a9ea-3436286bcdef None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:39.458363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e37c67-439e-44c4-a9ea-3436286bcdef None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:39.458634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e37c67-439e-44c4-a9ea-3436286bcdef None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:39.459424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e37c67-439e-44c4-a9ea-3436286bcdef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:39.459732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e37c67-439e-44c4-a9ea-3436286bcdef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:39.461221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e37c67-439e-44c4-a9ea-3436286bcdef None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:39.478336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729824-97c4-4727-ba5b-66b2b3d13f32 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:39.478627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729824-97c4-4727-ba5b-66b2b3d13f32 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:39.478908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729824-97c4-4727-ba5b-66b2b3d13f32 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:39.484688 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 392/788] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:39.485215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 397/789] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Aug 6 01:05:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:39.490479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace75840-5ae7-4f7e-a5f6-ed26220a2891 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:39.490822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace75840-5ae7-4f7e-a5f6-ed26220a2891 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:39.491126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e37c67-439e-44c4-a9ea-3436286bcdef None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:39.491327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace75840-5ae7-4f7e-a5f6-ed26220a2891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:39.530139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ace75840-5ae7-4f7e-a5f6-ed26220a2891 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:39.531498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ace75840-5ae7-4f7e-a5f6-ed26220a2891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:39.542971 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:39.603890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e37c67-439e-44c4-a9ea-3436286bcdef None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:39.662002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a326a4c-0ff7-469a-8296-b227deeb3b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:39.662969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a326a4c-0ff7-469a-8296-b227deeb3b25 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:39.663960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a326a4c-0ff7-469a-8296-b227deeb3b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:39.664378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a326a4c-0ff7-469a-8296-b227deeb3b25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:39.689058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a326a4c-0ff7-469a-8296-b227deeb3b25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:39.691348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a326a4c-0ff7-469a-8296-b227deeb3b25 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:39.720838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 393/790] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7a326a4c-0ff7-469a-8296-b227deeb3b25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:39.726920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:39.727865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:39.728778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:39.728778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:39.728778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:39.728778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:39.728778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 398/791] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:39.733928 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace75840-5ae7-4f7e-a5f6-ed26220a2891 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:39.748182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:39.749100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73237b10-efde-4192-b794-ae475cbd9f6a None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:39.749387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73237b10-efde-4192-b794-ae475cbd9f6a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:39.749661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73237b10-efde-4192-b794-ae475cbd9f6a None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:39.752596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 399/792] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:05:39.753330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73237b10-efde-4192-b794-ae475cbd9f6a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:39.851836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace75840-5ae7-4f7e-a5f6-ed26220a2891 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:39.880105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:39.890685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73237b10-efde-4192-b794-ae475cbd9f6a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:39.890854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2b544-8316-46a5-b53c-e5904fa9c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:39.891613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2b544-8316-46a5-b53c-e5904fa9c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:39.892050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2b544-8316-46a5-b53c-e5904fa9c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:39.892374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2b544-8316-46a5-b53c-e5904fa9c3c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:39.893178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2b544-8316-46a5-b53c-e5904fa9c3c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:39.893573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2b544-8316-46a5-b53c-e5904fa9c3c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:39.895028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2b544-8316-46a5-b53c-e5904fa9c3c6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:39.912291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 394/793] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:39.913855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45eb18-ca2d-44df-99ca-549ed7a83bee None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:39.914238 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45eb18-ca2d-44df-99ca-549ed7a83bee None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:39.914611 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45eb18-ca2d-44df-99ca-549ed7a83bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:39.933775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5df31-6743-451c-81db-3200b100abf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:39.934248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1e45eb18-ca2d-44df-99ca-549ed7a83bee None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:39.934871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5df31-6743-451c-81db-3200b100abf8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:39.935305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5df31-6743-451c-81db-3200b100abf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:39.935728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1e45eb18-ca2d-44df-99ca-549ed7a83bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:39.935887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5df31-6743-451c-81db-3200b100abf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:39.949988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5df31-6743-451c-81db-3200b100abf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7ff5df31-6743-451c-81db-3200b100abf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:39.956780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:39.957875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:39.958738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:39.964258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 400/794] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:39.965296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5df31-6743-451c-81db-3200b100abf8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.012903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:40.021667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 395/795] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:40.028790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45eb18-ca2d-44df-99ca-549ed7a83bee None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.089353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5df31-6743-451c-81db-3200b100abf8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:40.133694 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a12f7-e10d-4d5e-a1bd-836bf8276e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:40.134985 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a12f7-e10d-4d5e-a1bd-836bf8276e75 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.135378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a12f7-e10d-4d5e-a1bd-836bf8276e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.135762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a12f7-e10d-4d5e-a1bd-836bf8276e75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.145069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a12f7-e10d-4d5e-a1bd-836bf8276e75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:40.147132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a12f7-e10d-4d5e-a1bd-836bf8276e75 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-981a12f7-e10d-4d5e-a1bd-836bf8276e75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:40.162376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:40.164031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:40.165138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:40.165138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:40.165138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:40.165138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:40.165138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:40.165138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:40.165138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:40.165138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:40.166104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45eb18-ca2d-44df-99ca-549ed7a83bee None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:40.171566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 401/796] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:40.171566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a12f7-e10d-4d5e-a1bd-836bf8276e75 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.190703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:40.209674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5ca16-e8f4-485a-b862-fcbd378a1110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:40.210313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5ca16-e8f4-485a-b862-fcbd378a1110 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.210591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5ca16-e8f4-485a-b862-fcbd378a1110 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.210885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5ca16-e8f4-485a-b862-fcbd378a1110 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5ca16-e8f4-485a-b862-fcbd378a1110 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:40.211922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5ca16-e8f4-485a-b862-fcbd378a1110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:40.213313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5ca16-e8f4-485a-b862-fcbd378a1110 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:40.223961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 396/797] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:40.223961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5ca16-e8f4-485a-b862-fcbd378a1110 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.281236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:40.290318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a12f7-e10d-4d5e-a1bd-836bf8276e75 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:40.325545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5ca16-e8f4-485a-b862-fcbd378a1110 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:40.329630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbb053-c8bd-4acb-aa61-b2322d1a7fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:40.330291 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbb053-c8bd-4acb-aa61-b2322d1a7fea None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.330570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbb053-c8bd-4acb-aa61-b2322d1a7fea None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.330844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbb053-c8bd-4acb-aa61-b2322d1a7fea None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.331480 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbb053-c8bd-4acb-aa61-b2322d1a7fea None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:40.332908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbb053-c8bd-4acb-aa61-b2322d1a7fea None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-62bbb053-c8bd-4acb-aa61-b2322d1a7fea None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:40.342867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:40.343863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:40.344731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 402/798] 10.222.0.25 () {56 vars in 1129 bytes} [Sat Aug 6 01:05:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:40.346095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd82b4-1780-44d2-9b8e-ff5fe4279ecc None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.346095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd82b4-1780-44d2-9b8e-ff5fe4279ecc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.346095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd82b4-1780-44d2-9b8e-ff5fe4279ecc None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.351576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 403/799] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:40.352619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd82b4-1780-44d2-9b8e-ff5fe4279ecc None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.367536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d236a-237e-46d3-9c2c-3b864816dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:40.368194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d236a-237e-46d3-9c2c-3b864816dbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.368478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d236a-237e-46d3-9c2c-3b864816dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.368751 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d236a-237e-46d3-9c2c-3b864816dbe1 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.369503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d236a-237e-46d3-9c2c-3b864816dbe1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:40.369811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d236a-237e-46d3-9c2c-3b864816dbe1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:40.371233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d236a-237e-46d3-9c2c-3b864816dbe1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:40.386303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 397/800] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:40.387128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d236a-237e-46d3-9c2c-3b864816dbe1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.464372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:40.489827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd82b4-1780-44d2-9b8e-ff5fe4279ecc None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:40.490032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d236a-237e-46d3-9c2c-3b864816dbe1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:40.507477 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:40.525076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53025a50-819b-40da-9f6c-ce20c9ad1a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:40.525706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53025a50-819b-40da-9f6c-ce20c9ad1a2b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.527029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53025a50-819b-40da-9f6c-ce20c9ad1a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.527029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53025a50-819b-40da-9f6c-ce20c9ad1a2b None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.527029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53025a50-819b-40da-9f6c-ce20c9ad1a2b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:40.527458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53025a50-819b-40da-9f6c-ce20c9ad1a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:40.528711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53025a50-819b-40da-9f6c-ce20c9ad1a2b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:40.531681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8ab64-1198-460c-a276-4fe8c88c9ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:40.532343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8ab64-1198-460c-a276-4fe8c88c9ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.532621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8ab64-1198-460c-a276-4fe8c88c9ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.532894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8ab64-1198-460c-a276-4fe8c88c9ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.533483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 398/801] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Aug 6 01:05:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:40.534782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c324a4a-5446-4eed-b689-2e4b473ad39e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.535068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c324a4a-5446-4eed-b689-2e4b473ad39e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.535346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c324a4a-5446-4eed-b689-2e4b473ad39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.550305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8ab64-1198-460c-a276-4fe8c88c9ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:40.551754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8ab64-1198-460c-a276-4fe8c88c9ac2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:40.554601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9c324a4a-5446-4eed-b689-2e4b473ad39e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:40.555809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9c324a4a-5446-4eed-b689-2e4b473ad39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-19c8ab64-1198-460c-a276-4fe8c88c9ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:40.566887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:40.567819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:40.568751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:40.568751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:40.568751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:40.568751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:40.568751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 404/802] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:40.569047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8ab64-1198-460c-a276-4fe8c88c9ac2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.638336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 399/803] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:40.639552 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c324a4a-5446-4eed-b689-2e4b473ad39e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.686351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8ab64-1198-460c-a276-4fe8c88c9ac2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:40.717318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:40.721964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2c88-0582-4760-8214-a311fedfec7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:40.722665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2c88-0582-4760-8214-a311fedfec7a None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.722960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2c88-0582-4760-8214-a311fedfec7a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.723233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2c88-0582-4760-8214-a311fedfec7a None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/33aa626411f1474595102a4541af901a/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.726834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b2c88-0582-4760-8214-a311fedfec7a None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=33aa626411f1474595102a4541af901a, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:40.727321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b2c88-0582-4760-8214-a311fedfec7a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:40.771852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c324a4a-5446-4eed-b689-2e4b473ad39e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:40.785263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:40.794522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 405/804] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:05:40 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/33aa626411f1474595102a4541af901a/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:05:40.795352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2c88-0582-4760-8214-a311fedfec7a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.807735 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e4de0-a872-41cd-9f0f-5671286c6323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:40.808418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e4de0-a872-41cd-9f0f-5671286c6323 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.808722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e4de0-a872-41cd-9f0f-5671286c6323 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.809755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e4de0-a872-41cd-9f0f-5671286c6323 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.809755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e4de0-a872-41cd-9f0f-5671286c6323 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:40.810029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e4de0-a872-41cd-9f0f-5671286c6323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:40.811867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e4de0-a872-41cd-9f0f-5671286c6323 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:40.817762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 400/805] 10.222.0.25 () {56 vars in 1124 bytes} [Sat Aug 6 01:05:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:40.818633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e4de0-a872-41cd-9f0f-5671286c6323 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.917511 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2c88-0582-4760-8214-a311fedfec7a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:40.942983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e4de0-a872-41cd-9f0f-5671286c6323 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:40.958439 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e3793-07e0-4de4-badd-c2a86f0da7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:40.959083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e3793-07e0-4de4-badd-c2a86f0da7ae None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.959367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e3793-07e0-4de4-badd-c2a86f0da7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.959644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e3793-07e0-4de4-badd-c2a86f0da7ae None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.960949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e3793-07e0-4de4-badd-c2a86f0da7ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:40.961258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e3793-07e0-4de4-badd-c2a86f0da7ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:40.962644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e3793-07e0-4de4-badd-c2a86f0da7ae None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:40.975496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 406/806] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:40.977011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6ae2a-b0d0-4fc1-bef6-a36cf0a76cfb None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.977328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6ae2a-b0d0-4fc1-bef6-a36cf0a76cfb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.977610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6ae2a-b0d0-4fc1-bef6-a36cf0a76cfb None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:40.978825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 407/807] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:40.979570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6ae2a-b0d0-4fc1-bef6-a36cf0a76cfb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:40.993312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fee3f3-f762-4b90-aeb8-88964237f149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:40.994576 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fee3f3-f762-4b90-aeb8-88964237f149 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:40.995276 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fee3f3-f762-4b90-aeb8-88964237f149 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:40.995622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fee3f3-f762-4b90-aeb8-88964237f149 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:41.012299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fee3f3-f762-4b90-aeb8-88964237f149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-f1fee3f3-f762-4b90-aeb8-88964237f149 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:41.017265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:41.020792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 401/808] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:41.021868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fee3f3-f762-4b90-aeb8-88964237f149 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:41.053962 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:41.061654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:41.106615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6ae2a-b0d0-4fc1-bef6-a36cf0a76cfb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:41.130311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fee3f3-f762-4b90-aeb8-88964237f149 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:41.141659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2636ad-b803-495f-a16a-b7305d17570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:41.142292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2636ad-b803-495f-a16a-b7305d17570b None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:41.142579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2636ad-b803-495f-a16a-b7305d17570b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:41.142861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2636ad-b803-495f-a16a-b7305d17570b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:41.143571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2636ad-b803-495f-a16a-b7305d17570b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:41.144990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2636ad-b803-495f-a16a-b7305d17570b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:41.166513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8b69a-578d-45c0-af04-2960c86b61a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:41.167149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8b69a-578d-45c0-af04-2960c86b61a4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:41.167428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8b69a-578d-45c0-af04-2960c86b61a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:41.167699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8b69a-578d-45c0-af04-2960c86b61a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:41.174674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8b69a-578d-45c0-af04-2960c86b61a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:41.176167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8b69a-578d-45c0-af04-2960c86b61a4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-52e8b69a-578d-45c0-af04-2960c86b61a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:41.188068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:41.189237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:41.190230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 402/809] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Aug 6 01:05:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:41.190835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c24bdf-28d6-4f5e-bd46-e7aae5e50c90 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:41.190905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c24bdf-28d6-4f5e-bd46-e7aae5e50c90 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:41.191171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c24bdf-28d6-4f5e-bd46-e7aae5e50c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:41.209508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-95c24bdf-28d6-4f5e-bd46-e7aae5e50c90 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:41.210729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-95c24bdf-28d6-4f5e-bd46-e7aae5e50c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:41.251355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 408/810] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:05:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:41.253086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2636ad-b803-495f-a16a-b7305d17570b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:41.297418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 403/811] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:41.298673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c24bdf-28d6-4f5e-bd46-e7aae5e50c90 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:41.352255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:41.374029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2636ad-b803-495f-a16a-b7305d17570b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:41.384565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:41.412321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de51d02-d88b-4348-baab-e9266fc459d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:41.412935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de51d02-d88b-4348-baab-e9266fc459d9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:41.413437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de51d02-d88b-4348-baab-e9266fc459d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:41.413833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de51d02-d88b-4348-baab-e9266fc459d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:41.414671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de51d02-d88b-4348-baab-e9266fc459d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:41.415105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de51d02-d88b-4348-baab-e9266fc459d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:41.416679 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de51d02-d88b-4348-baab-e9266fc459d9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:41.431722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 409/812] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:41.432877 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de51d02-d88b-4348-baab-e9266fc459d9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:41.448056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c24bdf-28d6-4f5e-bd46-e7aae5e50c90 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:41.493879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc3b44-e230-4366-9e7d-2608d33f7713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:41.494866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc3b44-e230-4366-9e7d-2608d33f7713 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:41.495295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc3b44-e230-4366-9e7d-2608d33f7713 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:41.495732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc3b44-e230-4366-9e7d-2608d33f7713 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:41.496714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc3b44-e230-4366-9e7d-2608d33f7713 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:41.497174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc3b44-e230-4366-9e7d-2608d33f7713 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:41.498934 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc3b44-e230-4366-9e7d-2608d33f7713 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:41.523609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 404/813] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Aug 6 01:05:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:41.524766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc3b44-e230-4366-9e7d-2608d33f7713 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:41.551732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de51d02-d88b-4348-baab-e9266fc459d9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:41.599412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846bdb9-2e85-4796-af27-b0e89ff3e04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:41.600371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846bdb9-2e85-4796-af27-b0e89ff3e04a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:41.601397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846bdb9-2e85-4796-af27-b0e89ff3e04a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:41.601816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846bdb9-2e85-4796-af27-b0e89ff3e04a None admin] PATH_INFO: `/v3/services/network` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:41.602639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846bdb9-2e85-4796-af27-b0e89ff3e04a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:41.604189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846bdb9-2e85-4796-af27-b0e89ff3e04a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-c846bdb9-2e85-4796-af27-b0e89ff3e04a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:41.613803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:41.614854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 01:05:41.615722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:41.618592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 410/814] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:41 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:41.623475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846bdb9-2e85-4796-af27-b0e89ff3e04a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:41.630511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:41.649844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc3b44-e230-4366-9e7d-2608d33f7713 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:41.667097 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:41.704204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6525a5-1248-4d09-8f88-61c63e8a6ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:41.704851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6525a5-1248-4d09-8f88-61c63e8a6ebc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:41.705131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6525a5-1248-4d09-8f88-61c63e8a6ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:41.705403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6525a5-1248-4d09-8f88-61c63e8a6ebc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:41.717135 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6525a5-1248-4d09-8f88-61c63e8a6ebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:41.718580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6525a5-1248-4d09-8f88-61c63e8a6ebc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-6c6525a5-1248-4d09-8f88-61c63e8a6ebc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:41.735738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:41.736696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:41.737606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:41.737606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:41.737606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:41.737606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:41.737606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 405/815] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:41.738307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6525a5-1248-4d09-8f88-61c63e8a6ebc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:41.760891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846bdb9-2e85-4796-af27-b0e89ff3e04a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:41.806946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bdaf6-6ff3-4ca9-89ed-02f4e38f9365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:41.808489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bdaf6-6ff3-4ca9-89ed-02f4e38f9365 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:41.808774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bdaf6-6ff3-4ca9-89ed-02f4e38f9365 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:41.809048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bdaf6-6ff3-4ca9-89ed-02f4e38f9365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:41.849164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6525a5-1248-4d09-8f88-61c63e8a6ebc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:41.881384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:41.887460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69177910-193a-4d02-9916-cd33b1be9971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:41.888148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69177910-193a-4d02-9916-cd33b1be9971 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:41.888432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69177910-193a-4d02-9916-cd33b1be9971 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:41.888710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69177910-193a-4d02-9916-cd33b1be9971 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:41.889511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69177910-193a-4d02-9916-cd33b1be9971 None admin] RBAC: Adding query filter params (name=network) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:41.889829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69177910-193a-4d02-9916-cd33b1be9971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:41.891218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69177910-193a-4d02-9916-cd33b1be9971 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:41.896593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 406/816] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:05:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:41.897457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69177910-193a-4d02-9916-cd33b1be9971 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:41.900108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bdaf6-6ff3-4ca9-89ed-02f4e38f9365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:41.900108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bdaf6-6ff3-4ca9-89ed-02f4e38f9365 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:41.944462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:42.020096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 411/817] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:05:42.021578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bdaf6-6ff3-4ca9-89ed-02f4e38f9365 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:42.033092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69177910-193a-4d02-9916-cd33b1be9971 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:42.070118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495169c-4787-4f62-a14c-74e8e6977292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:42.071702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495169c-4787-4f62-a14c-74e8e6977292 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:42.071702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495169c-4787-4f62-a14c-74e8e6977292 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:42.071702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495169c-4787-4f62-a14c-74e8e6977292 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:42.072102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495169c-4787-4f62-a14c-74e8e6977292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:42.072398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495169c-4787-4f62-a14c-74e8e6977292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:42.073651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495169c-4787-4f62-a14c-74e8e6977292 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:42.087654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 407/818] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:42.088373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495169c-4787-4f62-a14c-74e8e6977292 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:42.148053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:42.165573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bdaf6-6ff3-4ca9-89ed-02f4e38f9365 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:42.196092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495169c-4787-4f62-a14c-74e8e6977292 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:42.205733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8137a7-1a61-424f-976b-60d1b8345a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:42.206357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8137a7-1a61-424f-976b-60d1b8345a16 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:42.206698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8137a7-1a61-424f-976b-60d1b8345a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:42.206983 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8137a7-1a61-424f-976b-60d1b8345a16 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:42.207666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8137a7-1a61-424f-976b-60d1b8345a16 None admin] RBAC: Adding query filter params (type=network) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:42.208017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8137a7-1a61-424f-976b-60d1b8345a16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:42.209405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8137a7-1a61-424f-976b-60d1b8345a16 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:42.211286 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:42.216302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 412/819] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:05:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:42.217266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8137a7-1a61-424f-976b-60d1b8345a16 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:42.242809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbd112-8da7-4a81-a86d-e6e9965b6b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:42.244294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbd112-8da7-4a81-a86d-e6e9965b6b63 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:42.244757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbd112-8da7-4a81-a86d-e6e9965b6b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:42.245212 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbd112-8da7-4a81-a86d-e6e9965b6b63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:42.259941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbd112-8da7-4a81-a86d-e6e9965b6b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:42.262083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbd112-8da7-4a81-a86d-e6e9965b6b63 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-6ccbd112-8da7-4a81-a86d-e6e9965b6b63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:42.283230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:42.284166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:42.285084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:42.285084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:42.285084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:42.285084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:42.288979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 408/820] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:42.289962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbd112-8da7-4a81-a86d-e6e9965b6b63 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:42.336599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8137a7-1a61-424f-976b-60d1b8345a16 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:42.382991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6b583-84ac-459d-b9c8-89c3d9ff8e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:42.384532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6b583-84ac-459d-b9c8-89c3d9ff8e24 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:42.385119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6b583-84ac-459d-b9c8-89c3d9ff8e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:42.385569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6b583-84ac-459d-b9c8-89c3d9ff8e24 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:42.396144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6b583-84ac-459d-b9c8-89c3d9ff8e24 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:42.399144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6b583-84ac-459d-b9c8-89c3d9ff8e24 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:42.407665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbd112-8da7-4a81-a86d-e6e9965b6b63 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b1d6b583-84ac-459d-b9c8-89c3d9ff8e24 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:42.414486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:42.415403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:42.416380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:42.416380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:42.416380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:42.416380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:42.416380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:42.416380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:42.416380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:42.416380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:42.416380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:05:42.416380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:42.417757 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 413/821] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Aug 6 01:05:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:42.418402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6b583-84ac-459d-b9c8-89c3d9ff8e24 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:42.420997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:42.448883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190619b-4f8c-4900-a4f7-492aba1d9f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:42.449384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190619b-4f8c-4900-a4f7-492aba1d9f66 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:42.449700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190619b-4f8c-4900-a4f7-492aba1d9f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:42.450002 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190619b-4f8c-4900-a4f7-492aba1d9f66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:42.450729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190619b-4f8c-4900-a4f7-492aba1d9f66 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:42.452262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190619b-4f8c-4900-a4f7-492aba1d9f66 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:42.492506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:42.539813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6b583-84ac-459d-b9c8-89c3d9ff8e24 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:42.540384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 409/822] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:05:42 2022] POST /identity/v3/endpoints => generated 337 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:42.541418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190619b-4f8c-4900-a4f7-492aba1d9f66 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:42.599811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efabc91-dbad-457f-97f6-4ec019430c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:42.599811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efabc91-dbad-457f-97f6-4ec019430c5d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:42.599811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efabc91-dbad-457f-97f6-4ec019430c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:42.599811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efabc91-dbad-457f-97f6-4ec019430c5d None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:42.600255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efabc91-dbad-457f-97f6-4ec019430c5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:42.600255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efabc91-dbad-457f-97f6-4ec019430c5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:42.600255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efabc91-dbad-457f-97f6-4ec019430c5d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:42.612325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 414/823] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:42.620958 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efabc91-dbad-457f-97f6-4ec019430c5d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:42.773946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190619b-4f8c-4900-a4f7-492aba1d9f66 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:42.795358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efabc91-dbad-457f-97f6-4ec019430c5d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:42.815714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:42.830981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ed3a3-d7e1-4d51-ba93-0e6e33a07f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:42.830981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ed3a3-d7e1-4d51-ba93-0e6e33a07f8d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:42.831905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ed3a3-d7e1-4d51-ba93-0e6e33a07f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:42.832215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ed3a3-d7e1-4d51-ba93-0e6e33a07f8d None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:42.834345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ed3a3-d7e1-4d51-ba93-0e6e33a07f8d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:42.834756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ed3a3-d7e1-4d51-ba93-0e6e33a07f8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:42.836418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ed3a3-d7e1-4d51-ba93-0e6e33a07f8d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:42.851408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d0c85-be51-476d-b1ca-154de88ebedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:42.852351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d0c85-be51-476d-b1ca-154de88ebedf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:42.852697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d0c85-be51-476d-b1ca-154de88ebedf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:42.852979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d0c85-be51-476d-b1ca-154de88ebedf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:42.859151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 410/824] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Aug 6 01:05:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:42.860507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2846c5-1d85-49cb-b364-a4a4edcc68e9 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:42.860794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2846c5-1d85-49cb-b364-a4a4edcc68e9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:42.861066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2846c5-1d85-49cb-b364-a4a4edcc68e9 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:42.866175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 411/825] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:42.866820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d0c85-be51-476d-b1ca-154de88ebedf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:42.866904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2846c5-1d85-49cb-b364-a4a4edcc68e9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:42.867003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d0c85-be51-476d-b1ca-154de88ebedf None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-5c7d0c85-be51-476d-b1ca-154de88ebedf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:42.879894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:42.880843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 415/826] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5d016-f5f8-45d1-b65a-ad6535bc9b19 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:42.887645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5d016-f5f8-45d1-b65a-ad6535bc9b19 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:42.888325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5d016-f5f8-45d1-b65a-ad6535bc9b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:42.899137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:42.906536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b9e5d016-f5f8-45d1-b65a-ad6535bc9b19 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:42.913339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b9e5d016-f5f8-45d1-b65a-ad6535bc9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:42.982367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 416/827] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:42.983209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5d016-f5f8-45d1-b65a-ad6535bc9b19 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:42.995295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2846c5-1d85-49cb-b364-a4a4edcc68e9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.031204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb316057-e7c8-4ce2-9c3e-5783b58103fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:43.031759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb316057-e7c8-4ce2-9c3e-5783b58103fc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.032270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb316057-e7c8-4ce2-9c3e-5783b58103fc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.032395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb316057-e7c8-4ce2-9c3e-5783b58103fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.044020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb316057-e7c8-4ce2-9c3e-5783b58103fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:43.045482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb316057-e7c8-4ce2-9c3e-5783b58103fc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-cb316057-e7c8-4ce2-9c3e-5783b58103fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:43.060935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:43.061860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:43.062768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:43.062768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:43.062768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:43.062768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:43.062768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 412/828] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:43.063047 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb316057-e7c8-4ce2-9c3e-5783b58103fc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:43.095451 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5d016-f5f8-45d1-b65a-ad6535bc9b19 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.098521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:43.131124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f1368-3b04-4eda-9b48-7d2ae2e8cc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:43.131753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f1368-3b04-4eda-9b48-7d2ae2e8cc4a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.132112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f1368-3b04-4eda-9b48-7d2ae2e8cc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.132573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f1368-3b04-4eda-9b48-7d2ae2e8cc4a None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.133332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f1368-3b04-4eda-9b48-7d2ae2e8cc4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:43.134026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f1368-3b04-4eda-9b48-7d2ae2e8cc4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:43.136060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f1368-3b04-4eda-9b48-7d2ae2e8cc4a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:43.148181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 417/829] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:43.149194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f1368-3b04-4eda-9b48-7d2ae2e8cc4a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:43.153932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:43.184281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb316057-e7c8-4ce2-9c3e-5783b58103fc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.220805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b4164-796e-4d12-9a2b-8fbbc4a4a516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:43.221473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b4164-796e-4d12-9a2b-8fbbc4a4a516 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.221759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b4164-796e-4d12-9a2b-8fbbc4a4a516 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.222031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b4164-796e-4d12-9a2b-8fbbc4a4a516 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.232905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b4164-796e-4d12-9a2b-8fbbc4a4a516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-356b4164-796e-4d12-9a2b-8fbbc4a4a516 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:43.238275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:43.239237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 413/830] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:43.244728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b4164-796e-4d12-9a2b-8fbbc4a4a516 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:43.256710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f1368-3b04-4eda-9b48-7d2ae2e8cc4a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.294668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcb8d3-22d1-4998-862e-d7bfdfeefb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:43.295518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcb8d3-22d1-4998-862e-d7bfdfeefb87 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.296010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcb8d3-22d1-4998-862e-d7bfdfeefb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.296454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcb8d3-22d1-4998-862e-d7bfdfeefb87 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.297728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcb8d3-22d1-4998-862e-d7bfdfeefb87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:43.298469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcb8d3-22d1-4998-862e-d7bfdfeefb87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:43.300506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcb8d3-22d1-4998-862e-d7bfdfeefb87 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:43.316877 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 418/831] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:43.317876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcb8d3-22d1-4998-862e-d7bfdfeefb87 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:43.346409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:43.347100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b4164-796e-4d12-9a2b-8fbbc4a4a516 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.382811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5bb5d-22e1-4c21-a566-54b7c065533a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:43.383429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5bb5d-22e1-4c21-a566-54b7c065533a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.383705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5bb5d-22e1-4c21-a566-54b7c065533a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.384027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5bb5d-22e1-4c21-a566-54b7c065533a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.384819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5bb5d-22e1-4c21-a566-54b7c065533a None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=db1b205d165543d0a3a69a2db6af5d1c) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:43.385138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5bb5d-22e1-4c21-a566-54b7c065533a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:43.386562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5bb5d-22e1-4c21-a566-54b7c065533a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:43.388393 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:43.403813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 414/832] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:43 2022] GET /identity/v3/role_assignments?user.id=db1b205d165543d0a3a69a2db6af5d1c&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:43.403813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5bb5d-22e1-4c21-a566-54b7c065533a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:43.449114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcb8d3-22d1-4998-862e-d7bfdfeefb87 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.489696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b0e1e-10f5-49ed-9e28-9687c50adf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:43.490338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b0e1e-10f5-49ed-9e28-9687c50adf9d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.490621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b0e1e-10f5-49ed-9e28-9687c50adf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.490902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b0e1e-10f5-49ed-9e28-9687c50adf9d None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.491813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b0e1e-10f5-49ed-9e28-9687c50adf9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:43.492148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b0e1e-10f5-49ed-9e28-9687c50adf9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:43.493636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b0e1e-10f5-49ed-9e28-9687c50adf9d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:43.501078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5bb5d-22e1-4c21-a566-54b7c065533a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.507576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 419/833] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:43.507576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d781a-5ad0-475b-94d8-d7b9b850f701 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.507576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d781a-5ad0-475b-94d8-d7b9b850f701 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.507576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d781a-5ad0-475b-94d8-d7b9b850f701 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.507576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 420/834] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:43.511580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d781a-5ad0-475b-94d8-d7b9b850f701 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:43.544060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54030f-6d96-4ee8-a1a0-9fa585dbef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:43.544834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54030f-6d96-4ee8-a1a0-9fa585dbef3f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.545003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54030f-6d96-4ee8-a1a0-9fa585dbef3f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.545286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54030f-6d96-4ee8-a1a0-9fa585dbef3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.552468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54030f-6d96-4ee8-a1a0-9fa585dbef3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:43.553944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54030f-6d96-4ee8-a1a0-9fa585dbef3f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bc54030f-6d96-4ee8-a1a0-9fa585dbef3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:43.567185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:43.568242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:43.569150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:43.569150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:43.569150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:43.569150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:43.569150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:43.569150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:43.569150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:43.569150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:43.569150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 415/835] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:43.570344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002fe89-5fa6-494e-bd3e-aa27d0a089b6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.570626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002fe89-5fa6-494e-bd3e-aa27d0a089b6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.570896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002fe89-5fa6-494e-bd3e-aa27d0a089b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.590062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f002fe89-5fa6-494e-bd3e-aa27d0a089b6 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:43.591264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f002fe89-5fa6-494e-bd3e-aa27d0a089b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:43.620617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:43.636573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d781a-5ad0-475b-94d8-d7b9b850f701 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.659008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:43.671159 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d570c4c-b192-4d79-98f8-af97824f621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:43.671828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d570c4c-b192-4d79-98f8-af97824f621a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.672112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d570c4c-b192-4d79-98f8-af97824f621a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.672382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d570c4c-b192-4d79-98f8-af97824f621a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.683853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 416/836] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:43.683853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002fe89-5fa6-494e-bd3e-aa27d0a089b6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:43.684096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d570c4c-b192-4d79-98f8-af97824f621a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:43.684096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d570c4c-b192-4d79-98f8-af97824f621a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-9d570c4c-b192-4d79-98f8-af97824f621a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:43.698873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:43.699827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:43.700747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:43.700747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:43.700747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:43.700747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:43.700747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 421/837] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:43.705722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d570c4c-b192-4d79-98f8-af97824f621a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:43.798233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002fe89-5fa6-494e-bd3e-aa27d0a089b6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.809562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d570c4c-b192-4d79-98f8-af97824f621a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.833306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e9883-6b3f-49d5-8f10-fb92a538cfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:43.833923 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9883-6b3f-49d5-8f10-fb92a538cfa4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.834203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9883-6b3f-49d5-8f10-fb92a538cfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.834477 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e9883-6b3f-49d5-8f10-fb92a538cfa4 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.835906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e9883-6b3f-49d5-8f10-fb92a538cfa4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:43.836218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e9883-6b3f-49d5-8f10-fb92a538cfa4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:43.837632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e9883-6b3f-49d5-8f10-fb92a538cfa4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:43.845229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f3f28-403f-4087-a759-9cf9fe839f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:43.845905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f3f28-403f-4087-a759-9cf9fe839f8f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:43.846166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f3f28-403f-4087-a759-9cf9fe839f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:43.846446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f3f28-403f-4087-a759-9cf9fe839f8f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:43.847071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f3f28-403f-4087-a759-9cf9fe839f8f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:43.848487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f3f28-403f-4087-a759-9cf9fe839f8f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:43.854221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 417/838] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:43.854221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e9883-6b3f-49d5-8f10-fb92a538cfa4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d19f3f28-403f-4087-a759-9cf9fe839f8f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:43.862778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:43.863809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:05:43.864685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:43.868728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 422/839] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:43.869543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f3f28-403f-4087-a759-9cf9fe839f8f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:43.872845 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:43.908131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:43.972792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e9883-6b3f-49d5-8f10-fb92a538cfa4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:43.977787 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f3f28-403f-4087-a759-9cf9fe839f8f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:44.011624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d61c88-7a73-409a-beba-d1014d0943bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:44.012280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d61c88-7a73-409a-beba-d1014d0943bb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:44.012567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d61c88-7a73-409a-beba-d1014d0943bb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:44.012846 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d61c88-7a73-409a-beba-d1014d0943bb None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:44.013130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c58d6-a023-4ac4-b9ef-9805990ce91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:44.013569 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d61c88-7a73-409a-beba-d1014d0943bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:44.013795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c58d6-a023-4ac4-b9ef-9805990ce91e None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:44.013895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d61c88-7a73-409a-beba-d1014d0943bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:44.014074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c58d6-a023-4ac4-b9ef-9805990ce91e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:44.014348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c58d6-a023-4ac4-b9ef-9805990ce91e None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/6d62ec1463c14665b94e9b2d85e48e14/roles/4dabc11cde5340ee9eb2c28e6154b3ec` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:44.015187 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d61c88-7a73-409a-beba-d1014d0943bb None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:44.029095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 418/840] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:44.029747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d61c88-7a73-409a-beba-d1014d0943bb None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:44.034366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c58d6-a023-4ac4-b9ef-9805990ce91e None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=6d62ec1463c14665b94e9b2d85e48e14, role_id=4dabc11cde5340ee9eb2c28e6154b3ec)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:44.035831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c58d6-a023-4ac4-b9ef-9805990ce91e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:44.113896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 423/841] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:05:43 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/6d62ec1463c14665b94e9b2d85e48e14/roles/4dabc11cde5340ee9eb2c28e6154b3ec => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:05:44.125020 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c58d6-a023-4ac4-b9ef-9805990ce91e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:44.128637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d61c88-7a73-409a-beba-d1014d0943bb None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:44.141868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:44.171355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a773121-c0c5-4a87-8930-4bc8d3253b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:44.172164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a773121-c0c5-4a87-8930-4bc8d3253b47 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:44.172494 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a773121-c0c5-4a87-8930-4bc8d3253b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:44.172769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a773121-c0c5-4a87-8930-4bc8d3253b47 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:44.173625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a773121-c0c5-4a87-8930-4bc8d3253b47 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:44.173994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a773121-c0c5-4a87-8930-4bc8d3253b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:44.175639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a773121-c0c5-4a87-8930-4bc8d3253b47 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:44.187237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 419/842] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:44.188287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a773121-c0c5-4a87-8930-4bc8d3253b47 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:44.213276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:44.283245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c58d6-a023-4ac4-b9ef-9805990ce91e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:44.324137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee4a24-f4a2-4efa-95e5-e59be6a9fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:44.325151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee4a24-f4a2-4efa-95e5-e59be6a9fbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:44.325263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a773121-c0c5-4a87-8930-4bc8d3253b47 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:44.325744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee4a24-f4a2-4efa-95e5-e59be6a9fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:44.326242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee4a24-f4a2-4efa-95e5-e59be6a9fbc9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:44.337057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee4a24-f4a2-4efa-95e5-e59be6a9fbc9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:44.339368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee4a24-f4a2-4efa-95e5-e59be6a9fbc9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-9cee4a24-f4a2-4efa-95e5-e59be6a9fbc9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:44.354665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:44.355670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:44.356676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:44.356676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:44.356676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:44.356676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:44.356676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:44.356676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:44.356676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:44.356676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:44.356676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 01:05:44.356676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:44.358042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 424/843] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:05:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:44.362467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee4a24-f4a2-4efa-95e5-e59be6a9fbc9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:44.374353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944e0f7-a3ec-4367-ae82-099ee9f33368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:44.375276 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944e0f7-a3ec-4367-ae82-099ee9f33368 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:44.375902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944e0f7-a3ec-4367-ae82-099ee9f33368 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:44.376388 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944e0f7-a3ec-4367-ae82-099ee9f33368 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:44.377384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944e0f7-a3ec-4367-ae82-099ee9f33368 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:44.377898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944e0f7-a3ec-4367-ae82-099ee9f33368 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:44.379562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944e0f7-a3ec-4367-ae82-099ee9f33368 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:44.385991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 420/844] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:44.479842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee4a24-f4a2-4efa-95e5-e59be6a9fbc9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:44.496165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:44.518877 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbb10b-afa1-4f58-a666-2927fd0e0733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:44.519527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbb10b-afa1-4f58-a666-2927fd0e0733 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:44.521723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbb10b-afa1-4f58-a666-2927fd0e0733 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:44.521723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbb10b-afa1-4f58-a666-2927fd0e0733 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:44.521723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbb10b-afa1-4f58-a666-2927fd0e0733 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:44.521723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbb10b-afa1-4f58-a666-2927fd0e0733 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:44.522816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbb10b-afa1-4f58-a666-2927fd0e0733 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:44.541876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 425/845] 10.222.0.25 () {56 vars in 1135 bytes} [Sat Aug 6 01:05:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:44.547535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944e0f7-a3ec-4367-ae82-099ee9f33368 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:44.595806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:44.658349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944e0f7-a3ec-4367-ae82-099ee9f33368 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:44.694570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280d12b-3e8e-408b-8c05-c0ddba19980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:44.695194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280d12b-3e8e-408b-8c05-c0ddba19980c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:44.695478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280d12b-3e8e-408b-8c05-c0ddba19980c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:44.695761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280d12b-3e8e-408b-8c05-c0ddba19980c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:44.765460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280d12b-3e8e-408b-8c05-c0ddba19980c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:44.766909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280d12b-3e8e-408b-8c05-c0ddba19980c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:44.800679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:44.868234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 421/846] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:05:44.873626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbb10b-afa1-4f58-a666-2927fd0e0733 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:44.978168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbb10b-afa1-4f58-a666-2927fd0e0733 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:45.011644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe4a26-838f-43b4-b9e2-a5935feaa240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:45.012322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe4a26-838f-43b4-b9e2-a5935feaa240 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:45.012606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe4a26-838f-43b4-b9e2-a5935feaa240 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:45.012884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe4a26-838f-43b4-b9e2-a5935feaa240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:45.023422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe4a26-838f-43b4-b9e2-a5935feaa240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:45.024915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe4a26-838f-43b4-b9e2-a5935feaa240 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e7fe4a26-838f-43b4-b9e2-a5935feaa240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:45.040027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:45.040914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:45.041814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:45.041814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:45.041814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:45.041814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:45.041814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 426/847] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:45.047081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280d12b-3e8e-408b-8c05-c0ddba19980c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:45.152739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280d12b-3e8e-408b-8c05-c0ddba19980c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:45.190941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ca9e3-2467-4787-8e09-058e6c470387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:45.191981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ca9e3-2467-4787-8e09-058e6c470387 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:45.192451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ca9e3-2467-4787-8e09-058e6c470387 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:45.192925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ca9e3-2467-4787-8e09-058e6c470387 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:45.193813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ca9e3-2467-4787-8e09-058e6c470387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:45.194374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ca9e3-2467-4787-8e09-058e6c470387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:45.195966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ca9e3-2467-4787-8e09-058e6c470387 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:45.198885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:45.209384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 422/848] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:45.215650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe4a26-838f-43b4-b9e2-a5935feaa240 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:45.242323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:45.324417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe4a26-838f-43b4-b9e2-a5935feaa240 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:45.361727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6658f8-a5c6-4399-a041-917c39d22803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:45.362373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6658f8-a5c6-4399-a041-917c39d22803 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:45.362650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6658f8-a5c6-4399-a041-917c39d22803 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:45.362920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6658f8-a5c6-4399-a041-917c39d22803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:45.369806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6658f8-a5c6-4399-a041-917c39d22803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:45.371249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6658f8-a5c6-4399-a041-917c39d22803 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-7c6658f8-a5c6-4399-a041-917c39d22803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:45.383828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:45.384732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:45.385680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:45.385680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:45.385680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:45.385680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:45.385680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:45.385680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:45.385680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:45.385680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:45.385680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 427/849] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:45.392419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ca9e3-2467-4787-8e09-058e6c470387 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:45.497720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ca9e3-2467-4787-8e09-058e6c470387 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:45.506029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c3270-0258-4227-b2ca-6fe31648e534 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:45.506320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c3270-0258-4227-b2ca-6fe31648e534 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:45.506605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c3270-0258-4227-b2ca-6fe31648e534 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:45.512471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 428/850] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:45.514879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821323f0-56ec-44ba-8a63-d420c1227e7d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:45.515319 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821323f0-56ec-44ba-8a63-d420c1227e7d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:45.515606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821323f0-56ec-44ba-8a63-d420c1227e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:45.534362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-821323f0-56ec-44ba-8a63-d420c1227e7d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:45.535746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-821323f0-56ec-44ba-8a63-d420c1227e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:45.537241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa42979-9448-4438-b3e7-1c645222268a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:45.538259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42979-9448-4438-b3e7-1c645222268a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:45.538875 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42979-9448-4438-b3e7-1c645222268a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:45.539161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa42979-9448-4438-b3e7-1c645222268a None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:45.539954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa42979-9448-4438-b3e7-1c645222268a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:45.540272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa42979-9448-4438-b3e7-1c645222268a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:45.541701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa42979-9448-4438-b3e7-1c645222268a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:45.554628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 423/851] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:45.560637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa42979-9448-4438-b3e7-1c645222268a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:45.621478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 429/852] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:05:45.642985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:45.643633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821323f0-56ec-44ba-8a63-d420c1227e7d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:45.667080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:45.696981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa42979-9448-4438-b3e7-1c645222268a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:45.734197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ca6f0-0af0-4cd3-8ea3-0bb93424e878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:45.734993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ca6f0-0af0-4cd3-8ea3-0bb93424e878 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:45.735893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ca6f0-0af0-4cd3-8ea3-0bb93424e878 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:45.736217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ca6f0-0af0-4cd3-8ea3-0bb93424e878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:45.737032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ca6f0-0af0-4cd3-8ea3-0bb93424e878 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=33aa626411f1474595102a4541af901a) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:45.737356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ca6f0-0af0-4cd3-8ea3-0bb93424e878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:45.738790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ca6f0-0af0-4cd3-8ea3-0bb93424e878 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:45.751225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 424/853] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:45 2022] GET /identity/v3/role_assignments?user.id=33aa626411f1474595102a4541af901a&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:45.777470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821323f0-56ec-44ba-8a63-d420c1227e7d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:45.815335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bcbf8-7bdc-4c77-9dfd-05b09c53d603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:45.816033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bcbf8-7bdc-4c77-9dfd-05b09c53d603 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:45.816320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bcbf8-7bdc-4c77-9dfd-05b09c53d603 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:45.816595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bcbf8-7bdc-4c77-9dfd-05b09c53d603 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:45.817245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bcbf8-7bdc-4c77-9dfd-05b09c53d603 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:45.818658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bcbf8-7bdc-4c77-9dfd-05b09c53d603 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-c48bcbf8-7bdc-4c77-9dfd-05b09c53d603 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:45.829593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:45.830618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:05:45.831471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:45.836083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 430/854] 10.222.0.25 () {56 vars in 1132 bytes} [Sat Aug 6 01:05:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:45.837593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ca6f0-0af0-4cd3-8ea3-0bb93424e878 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:45.950471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ca6f0-0af0-4cd3-8ea3-0bb93424e878 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:45.971123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:45.985239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449249ac-7e86-4921-9d31-91caf732fcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:45.985871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449249ac-7e86-4921-9d31-91caf732fcf5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:45.986154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449249ac-7e86-4921-9d31-91caf732fcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:45.986433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449249ac-7e86-4921-9d31-91caf732fcf5 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:45.996009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449249ac-7e86-4921-9d31-91caf732fcf5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:45.996009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449249ac-7e86-4921-9d31-91caf732fcf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:45.996009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449249ac-7e86-4921-9d31-91caf732fcf5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:45.998007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 425/855] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:05:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:46.000175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bcbf8-7bdc-4c77-9dfd-05b09c53d603 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:46.081991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:46.105880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bcbf8-7bdc-4c77-9dfd-05b09c53d603 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:46.140411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252df7e-cee0-4897-8bf0-933beb3dceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:46.141047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252df7e-cee0-4897-8bf0-933beb3dceed None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:46.141329 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252df7e-cee0-4897-8bf0-933beb3dceed None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:46.141606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252df7e-cee0-4897-8bf0-933beb3dceed None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:46.142293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252df7e-cee0-4897-8bf0-933beb3dceed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:46.142601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252df7e-cee0-4897-8bf0-933beb3dceed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:46.143998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252df7e-cee0-4897-8bf0-933beb3dceed None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:46.152346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 431/856] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:05:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:46.284563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a22cd9-75ba-4948-8e08-97458675d2c2 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:46.285109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a22cd9-75ba-4948-8e08-97458675d2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:46.285109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a22cd9-75ba-4948-8e08-97458675d2c2 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:46.286860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 426/857] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:46.293548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab920ca2-18bb-436c-91f2-e4b0fca42175 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:46.294066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab920ca2-18bb-436c-91f2-e4b0fca42175 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:46.294501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab920ca2-18bb-436c-91f2-e4b0fca42175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:46.314408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ab920ca2-18bb-436c-91f2-e4b0fca42175 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:46.315628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ab920ca2-18bb-436c-91f2-e4b0fca42175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:46.385609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 432/858] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:46.408280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a22cd9-75ba-4948-8e08-97458675d2c2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:46.506584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a554-1e0d-4df6-9ede-0a61ea4ca88f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:46.506938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a554-1e0d-4df6-9ede-0a61ea4ca88f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:46.507166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a554-1e0d-4df6-9ede-0a61ea4ca88f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:46.508761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 433/859] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:46.513799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c566ff-9de8-4dfd-a268-fcdb1a0642b7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:46.514074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c566ff-9de8-4dfd-a268-fcdb1a0642b7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:46.514352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c566ff-9de8-4dfd-a268-fcdb1a0642b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:46.515591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a22cd9-75ba-4948-8e08-97458675d2c2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:46.534143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-73c566ff-9de8-4dfd-a268-fcdb1a0642b7 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:46.535624 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-73c566ff-9de8-4dfd-a268-fcdb1a0642b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:46.555459 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b395e3b-e94b-4074-a57f-77177c9f805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:46.556698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b395e3b-e94b-4074-a57f-77177c9f805c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:46.557196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b395e3b-e94b-4074-a57f-77177c9f805c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:46.558010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b395e3b-e94b-4074-a57f-77177c9f805c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:46.572603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b395e3b-e94b-4074-a57f-77177c9f805c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-6b395e3b-e94b-4074-a57f-77177c9f805c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:46.581840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:46.582789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:46.583637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 427/860] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:05:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:46.592897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b395e3b-e94b-4074-a57f-77177c9f805c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:46.628070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:46.648954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 434/861] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:05:46.658642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:46.661101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c566ff-9de8-4dfd-a268-fcdb1a0642b7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:46.713424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b395e3b-e94b-4074-a57f-77177c9f805c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:46.748274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4757004-49b7-4780-8785-54a0a6bb9566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:46.749045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4757004-49b7-4780-8785-54a0a6bb9566 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:46.749321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4757004-49b7-4780-8785-54a0a6bb9566 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:46.749604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4757004-49b7-4780-8785-54a0a6bb9566 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:46.750381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4757004-49b7-4780-8785-54a0a6bb9566 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:46.750792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4757004-49b7-4780-8785-54a0a6bb9566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:46.752115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4757004-49b7-4780-8785-54a0a6bb9566 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:46.761084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 428/862] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:05:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:46.766167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4757004-49b7-4780-8785-54a0a6bb9566 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:46.766671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c566ff-9de8-4dfd-a268-fcdb1a0642b7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:46.804700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1ab35-41e7-4918-ab34-6a698adfcfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:46.805606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1ab35-41e7-4918-ab34-6a698adfcfc6 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:46.806198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1ab35-41e7-4918-ab34-6a698adfcfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:46.806758 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1ab35-41e7-4918-ab34-6a698adfcfc6 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:46.807665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1ab35-41e7-4918-ab34-6a698adfcfc6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:46.809260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1ab35-41e7-4918-ab34-6a698adfcfc6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:46.879107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4757004-49b7-4780-8785-54a0a6bb9566 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:46.887568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:46.898246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 435/863] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:05:46 2022] POST /identity/v3/services => generated 251 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:05:46.932231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f032524-b5d9-4917-b8ae-5b866e1c9bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:46.933279 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f032524-b5d9-4917-b8ae-5b866e1c9bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:46.933775 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f032524-b5d9-4917-b8ae-5b866e1c9bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:46.934272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f032524-b5d9-4917-b8ae-5b866e1c9bf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:46.950708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f032524-b5d9-4917-b8ae-5b866e1c9bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:46.950708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f032524-b5d9-4917-b8ae-5b866e1c9bf5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-8f032524-b5d9-4917-b8ae-5b866e1c9bf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:46.967729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:46.968732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:46.969732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:46.969732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:46.969732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:46.969732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:46.969732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 429/864] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:46.976044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1ab35-41e7-4918-ab34-6a698adfcfc6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:47.045544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:47.090402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1ab35-41e7-4918-ab34-6a698adfcfc6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:47.124974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3f448-d84f-438f-b183-c504064248ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:47.125996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3f448-d84f-438f-b183-c504064248ec None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:47.126482 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3f448-d84f-438f-b183-c504064248ec None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:47.126970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3f448-d84f-438f-b183-c504064248ec None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:47.127875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3f448-d84f-438f-b183-c504064248ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:47.128395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3f448-d84f-438f-b183-c504064248ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:47.130029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3f448-d84f-438f-b183-c504064248ec None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:47.142534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 436/865] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:47.148850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f032524-b5d9-4917-b8ae-5b866e1c9bf5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:47.250886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f032524-b5d9-4917-b8ae-5b866e1c9bf5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:47.285612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59417ee3-0e08-40eb-9a9b-f1bcbddbac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:47.286242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59417ee3-0e08-40eb-9a9b-f1bcbddbac88 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:47.286515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59417ee3-0e08-40eb-9a9b-f1bcbddbac88 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:47.286789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59417ee3-0e08-40eb-9a9b-f1bcbddbac88 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:47.295567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59417ee3-0e08-40eb-9a9b-f1bcbddbac88 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:47.297041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59417ee3-0e08-40eb-9a9b-f1bcbddbac88 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-59417ee3-0e08-40eb-9a9b-f1bcbddbac88 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:47.309025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:47.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:47.311000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 430/866] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Aug 6 01:05:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:47.317999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3f448-d84f-438f-b183-c504064248ec None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:47.426979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3f448-d84f-438f-b183-c504064248ec None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:47.453686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:47.468922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56b2e5-4393-444a-9239-875659d2e275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:47.472333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56b2e5-4393-444a-9239-875659d2e275 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:47.472333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56b2e5-4393-444a-9239-875659d2e275 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:47.472333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56b2e5-4393-444a-9239-875659d2e275 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:47.472333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56b2e5-4393-444a-9239-875659d2e275 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:47.472333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56b2e5-4393-444a-9239-875659d2e275 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:47.473580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56b2e5-4393-444a-9239-875659d2e275 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:47.492987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 437/867] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Aug 6 01:05:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:47.500840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59417ee3-0e08-40eb-9a9b-f1bcbddbac88 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:47.517706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:47.603104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59417ee3-0e08-40eb-9a9b-f1bcbddbac88 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:47.638345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdcde9-85d4-4406-927b-5caec339aa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:47.638962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdcde9-85d4-4406-927b-5caec339aa5d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:47.639230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdcde9-85d4-4406-927b-5caec339aa5d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:47.639503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdcde9-85d4-4406-927b-5caec339aa5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:47.712348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdcde9-85d4-4406-927b-5caec339aa5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:47.713798 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdcde9-85d4-4406-927b-5caec339aa5d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:47.729233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:47.814100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 431/868] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:47 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:05:47.819769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56b2e5-4393-444a-9239-875659d2e275 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:47.926625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56b2e5-4393-444a-9239-875659d2e275 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:47.963267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4110da3-983e-4e01-8840-adb706797498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:47.964172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4110da3-983e-4e01-8840-adb706797498 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:47.964602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4110da3-983e-4e01-8840-adb706797498 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:47.965052 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4110da3-983e-4e01-8840-adb706797498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:47.975531 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4110da3-983e-4e01-8840-adb706797498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:47.977846 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4110da3-983e-4e01-8840-adb706797498 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:47.988123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a5534-158f-426f-b098-e9c7f7a84ff5 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:47.988907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a5534-158f-426f-b098-e9c7f7a84ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:47.990187 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a5534-158f-426f-b098-e9c7f7a84ff5 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:47.993861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 432/869] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e4110da3-983e-4e01-8840-adb706797498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:47.999236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:48.000637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:48.002070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:48.002070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:48.002070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:48.002070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:48.002070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 438/870] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:48.004919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b142571-66a6-436b-a458-23b6776a2afc None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.005340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b142571-66a6-436b-a458-23b6776a2afc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.005746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b142571-66a6-436b-a458-23b6776a2afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.018079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4110da3-983e-4e01-8840-adb706797498 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:48.038883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5b142571-66a6-436b-a458-23b6776a2afc None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:48.040762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5b142571-66a6-436b-a458-23b6776a2afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:48.052671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:48.121049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 433/871] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:48 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:48.128255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4110da3-983e-4e01-8840-adb706797498 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:48.142747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b142571-66a6-436b-a458-23b6776a2afc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:48.175011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172932c-5e43-4b33-b8e2-54d3506bd32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:48.176592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172932c-5e43-4b33-b8e2-54d3506bd32f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.177339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172932c-5e43-4b33-b8e2-54d3506bd32f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.177814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172932c-5e43-4b33-b8e2-54d3506bd32f None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.178767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172932c-5e43-4b33-b8e2-54d3506bd32f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:48.179533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172932c-5e43-4b33-b8e2-54d3506bd32f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:48.182090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172932c-5e43-4b33-b8e2-54d3506bd32f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:48.194088 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:48.199526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 439/872] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:48.205639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172932c-5e43-4b33-b8e2-54d3506bd32f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:48.258271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b142571-66a6-436b-a458-23b6776a2afc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:48.293902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca786f0d-eec2-4e38-b027-c95966d321e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:48.294547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca786f0d-eec2-4e38-b027-c95966d321e4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.294838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca786f0d-eec2-4e38-b027-c95966d321e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.295119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca786f0d-eec2-4e38-b027-c95966d321e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.295764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca786f0d-eec2-4e38-b027-c95966d321e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:48.297158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca786f0d-eec2-4e38-b027-c95966d321e4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-ca786f0d-eec2-4e38-b027-c95966d321e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:48.306563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:48.307607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:48.308511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 434/873] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:05:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:48.309337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ab90c-9547-480b-a3c2-48be6ac8d64c None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.309337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ab90c-9547-480b-a3c2-48be6ac8d64c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.309471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ab90c-9547-480b-a3c2-48be6ac8d64c None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.311823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 435/874] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:48.313337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ab90c-9547-480b-a3c2-48be6ac8d64c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:48.320958 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:48.331926 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172932c-5e43-4b33-b8e2-54d3506bd32f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:48.366291 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e67b07-3d59-45bf-a6e8-872ab15c378e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:48.367354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e67b07-3d59-45bf-a6e8-872ab15c378e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.367977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e67b07-3d59-45bf-a6e8-872ab15c378e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.368438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e67b07-3d59-45bf-a6e8-872ab15c378e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.375836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e67b07-3d59-45bf-a6e8-872ab15c378e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:48.377462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e67b07-3d59-45bf-a6e8-872ab15c378e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d6e67b07-3d59-45bf-a6e8-872ab15c378e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:48.389544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:48.390464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:48.391379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:48.391379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:48.391379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:48.391379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:48.391379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:48.391379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:48.391379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:48.391379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:48.395353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 440/875] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:48.396662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d00113-3a10-4b72-b239-77521bc95409 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.396951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d00113-3a10-4b72-b239-77521bc95409 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.397230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d00113-3a10-4b72-b239-77521bc95409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.416452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-67d00113-3a10-4b72-b239-77521bc95409 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:48.420366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-67d00113-3a10-4b72-b239-77521bc95409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:48.424361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ab90c-9547-480b-a3c2-48be6ac8d64c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:48.435511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:48.462806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c885c-9234-4a54-83c9-8491501c6114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:48.463434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c885c-9234-4a54-83c9-8491501c6114 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.463717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c885c-9234-4a54-83c9-8491501c6114 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.464025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c885c-9234-4a54-83c9-8491501c6114 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.464735 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c885c-9234-4a54-83c9-8491501c6114 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:48.465048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c885c-9234-4a54-83c9-8491501c6114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:48.466406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c885c-9234-4a54-83c9-8491501c6114 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:48.475996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 436/876] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Aug 6 01:05:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:48.475996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c885c-9234-4a54-83c9-8491501c6114 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:48.503799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 441/877] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:48 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:48.503799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d00113-3a10-4b72-b239-77521bc95409 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:48.586696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:48.598268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c885c-9234-4a54-83c9-8491501c6114 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:48.608406 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d00113-3a10-4b72-b239-77521bc95409 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:48.636026 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c62dc-07eb-43b8-ad41-db8427479421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:48.636563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c62dc-07eb-43b8-ad41-db8427479421 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.636847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c62dc-07eb-43b8-ad41-db8427479421 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.637126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c62dc-07eb-43b8-ad41-db8427479421 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.637881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c62dc-07eb-43b8-ad41-db8427479421 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:48.638197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c62dc-07eb-43b8-ad41-db8427479421 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:48.639634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c62dc-07eb-43b8-ad41-db8427479421 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:48.645117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ced1f-69bb-4618-a845-372b41414843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:48.645742 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ced1f-69bb-4618-a845-372b41414843 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.646019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ced1f-69bb-4618-a845-372b41414843 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.646290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ced1f-69bb-4618-a845-372b41414843 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.646970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ced1f-69bb-4618-a845-372b41414843 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:48.647273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ced1f-69bb-4618-a845-372b41414843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:48.648701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ced1f-69bb-4618-a845-372b41414843 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:48.651964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 437/878] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:48.652902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c62dc-07eb-43b8-ad41-db8427479421 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:48.664071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 442/879] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Aug 6 01:05:48 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:48.665067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ced1f-69bb-4618-a845-372b41414843 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:48.699896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:48.792624 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ced1f-69bb-4618-a845-372b41414843 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:48.811983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c62dc-07eb-43b8-ad41-db8427479421 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:48.834903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffc249-335e-4c47-9d8d-ee96efffd837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:48.836394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffc249-335e-4c47-9d8d-ee96efffd837 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.837057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffc249-335e-4c47-9d8d-ee96efffd837 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.838310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffc249-335e-4c47-9d8d-ee96efffd837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.839393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffc249-335e-4c47-9d8d-ee96efffd837 None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=6d62ec1463c14665b94e9b2d85e48e14) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:48.840060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffc249-335e-4c47-9d8d-ee96efffd837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:48.841829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffc249-335e-4c47-9d8d-ee96efffd837 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:48.855809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 443/880] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:48 2022] GET /identity/v3/role_assignments?user.id=6d62ec1463c14665b94e9b2d85e48e14&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:48.858063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1051c9b-9e0c-4f74-84c1-95aab6ad8359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:48.858719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1051c9b-9e0c-4f74-84c1-95aab6ad8359 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:48.859004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1051c9b-9e0c-4f74-84c1-95aab6ad8359 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:48.859287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1051c9b-9e0c-4f74-84c1-95aab6ad8359 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:48.860085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1051c9b-9e0c-4f74-84c1-95aab6ad8359 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:48.861590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1051c9b-9e0c-4f74-84c1-95aab6ad8359 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:48.965490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 438/881] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:05:48 2022] POST /identity/v3/services => generated 269 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:49.055682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d838a-577d-42de-8522-b885cf1e08d9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:49.055999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d838a-577d-42de-8522-b885cf1e08d9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:49.056281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d838a-577d-42de-8522-b885cf1e08d9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:49.057988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 444/882] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:49.063653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb845-e7f8-4c31-a1c1-1889f702abea None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:49.064096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb845-e7f8-4c31-a1c1-1889f702abea None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:49.064385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb845-e7f8-4c31-a1c1-1889f702abea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:49.087123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-73ecb845-e7f8-4c31-a1c1-1889f702abea None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:49.088491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-73ecb845-e7f8-4c31-a1c1-1889f702abea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:49.169107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 439/883] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:49.192914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d838a-577d-42de-8522-b885cf1e08d9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:49.206506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:49.305012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d838a-577d-42de-8522-b885cf1e08d9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:49.339192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0924dc-b7ff-4e38-ab49-75f246a0eb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:49.339916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0924dc-b7ff-4e38-ab49-75f246a0eb4d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:49.340197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0924dc-b7ff-4e38-ab49-75f246a0eb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:49.340468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0924dc-b7ff-4e38-ab49-75f246a0eb4d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:49.341090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0924dc-b7ff-4e38-ab49-75f246a0eb4d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:49.342456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0924dc-b7ff-4e38-ab49-75f246a0eb4d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0d0924dc-b7ff-4e38-ab49-75f246a0eb4d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:49.351956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:49.352929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:49.353917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 445/884] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:49.364905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecb845-e7f8-4c31-a1c1-1889f702abea None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:49.385503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:49.473476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecb845-e7f8-4c31-a1c1-1889f702abea None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:49.505735 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07510e20-f674-40d3-87f4-c3d238ce920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:49.506359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07510e20-f674-40d3-87f4-c3d238ce920e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:49.506642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07510e20-f674-40d3-87f4-c3d238ce920e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:49.506947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07510e20-f674-40d3-87f4-c3d238ce920e None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:49.507628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07510e20-f674-40d3-87f4-c3d238ce920e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:49.507962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07510e20-f674-40d3-87f4-c3d238ce920e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:49.509377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07510e20-f674-40d3-87f4-c3d238ce920e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:49.514367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 440/885] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:49.521135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0924dc-b7ff-4e38-ab49-75f246a0eb4d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:49.596498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:49.626641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0924dc-b7ff-4e38-ab49-75f246a0eb4d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:49.663110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80135865-339f-4e1c-a87c-bd372f6e29f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:49.663746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80135865-339f-4e1c-a87c-bd372f6e29f9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:49.664063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80135865-339f-4e1c-a87c-bd372f6e29f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:49.664340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80135865-339f-4e1c-a87c-bd372f6e29f9 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:49.665018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80135865-339f-4e1c-a87c-bd372f6e29f9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:49.665325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80135865-339f-4e1c-a87c-bd372f6e29f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:49.666699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80135865-339f-4e1c-a87c-bd372f6e29f9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:49.672007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 446/886] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:49.677902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07510e20-f674-40d3-87f4-c3d238ce920e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:49.761491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:49.779880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07510e20-f674-40d3-87f4-c3d238ce920e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:49.819562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ffc61-ff83-4181-ac62-f9097f3ab6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:49.820344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ffc61-ff83-4181-ac62-f9097f3ab6ac None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:49.820632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ffc61-ff83-4181-ac62-f9097f3ab6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:49.820910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ffc61-ff83-4181-ac62-f9097f3ab6ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:49.821703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ffc61-ff83-4181-ac62-f9097f3ab6ac None admin] RBAC: Adding query filter params (interface=public, service_id=3d2ebe521c6649da8659b5d2f0a70172, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:49.822018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ffc61-ff83-4181-ac62-f9097f3ab6ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:49.823550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ffc61-ff83-4181-ac62-f9097f3ab6ac None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:49.830283 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 441/887] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Aug 6 01:05:49 2022] GET /identity/v3/endpoints?service_id=3d2ebe521c6649da8659b5d2f0a70172&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:49.836030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80135865-339f-4e1c-a87c-bd372f6e29f9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:49.938716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80135865-339f-4e1c-a87c-bd372f6e29f9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:49.972093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a5875-aea0-4025-9034-e83b97ceaa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:49.972715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a5875-aea0-4025-9034-e83b97ceaa3b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:49.973044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a5875-aea0-4025-9034-e83b97ceaa3b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:49.973321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a5875-aea0-4025-9034-e83b97ceaa3b None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:49.973975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a5875-aea0-4025-9034-e83b97ceaa3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:49.975342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a5875-aea0-4025-9034-e83b97ceaa3b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:49.977648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:49.982345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 447/888] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Aug 6 01:05:49 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:05:50.759930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab68a17-6378-4664-b18b-9d5b0f517e8b None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:50.760472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab68a17-6378-4664-b18b-9d5b0f517e8b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:50.760472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab68a17-6378-4664-b18b-9d5b0f517e8b None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:50.762116 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 442/889] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:50.767252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1757f5-7a01-4e80-922f-8e4b8e18d7f1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:50.768100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1757f5-7a01-4e80-922f-8e4b8e18d7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:50.768587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1757f5-7a01-4e80-922f-8e4b8e18d7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:50.787368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ec1757f5-7a01-4e80-922f-8e4b8e18d7f1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:50.788935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ec1757f5-7a01-4e80-922f-8e4b8e18d7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:50.858340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 448/890] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:50.881146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab68a17-6378-4664-b18b-9d5b0f517e8b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:50.987488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab68a17-6378-4664-b18b-9d5b0f517e8b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:51.028472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096b3f7-ad36-40f0-a860-f9e29f76926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:51.029145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096b3f7-ad36-40f0-a860-f9e29f76926b None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.029427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096b3f7-ad36-40f0-a860-f9e29f76926b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.029699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096b3f7-ad36-40f0-a860-f9e29f76926b None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.030406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096b3f7-ad36-40f0-a860-f9e29f76926b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:51.031856 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096b3f7-ad36-40f0-a860-f9e29f76926b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:51.055990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184341-48f6-4d4c-844c-a149aaac4ae0 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.056303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184341-48f6-4d4c-844c-a149aaac4ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.056596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184341-48f6-4d4c-844c-a149aaac4ae0 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.058084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 449/891] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:05:51.063100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6d58-5466-4c05-9af9-59333feea944 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.063389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6d58-5466-4c05-9af9-59333feea944 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.063667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6d58-5466-4c05-9af9-59333feea944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.082815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-21de6d58-5466-4c05-9af9-59333feea944 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:51.084035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-21de6d58-5466-4c05-9af9-59333feea944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:51.095240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 443/892] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:05:50 2022] POST /identity/v3/services => generated 241 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:51.096744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a95c88-ebeb-47e9-af32-4f04faac482f None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.097088 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a95c88-ebeb-47e9-af32-4f04faac482f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.097371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a95c88-ebeb-47e9-af32-4f04faac482f None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.099194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 444/893] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:51.105885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ca5f-465c-41ff-81c2-7e15ed7a3b43 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.110703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ca5f-465c-41ff-81c2-7e15ed7a3b43 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.110703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ca5f-465c-41ff-81c2-7e15ed7a3b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.122473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:51.128861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f4e8ca5f-465c-41ff-81c2-7e15ed7a3b43 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:51.130447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f4e8ca5f-465c-41ff-81c2-7e15ed7a3b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:51.200244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 450/894] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:05:51.216799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 445/895] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:51.223325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6d58-5466-4c05-9af9-59333feea944 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:51.244706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8ca5f-465c-41ff-81c2-7e15ed7a3b43 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:51.294151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:51.354441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6d58-5466-4c05-9af9-59333feea944 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:51.368659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8ca5f-465c-41ff-81c2-7e15ed7a3b43 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:51.400015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adcf16-80b5-4951-a736-e59c6de89f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:51.401074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adcf16-80b5-4951-a736-e59c6de89f0e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.401357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adcf16-80b5-4951-a736-e59c6de89f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.402029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adcf16-80b5-4951-a736-e59c6de89f0e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.406767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed363d77-7e30-4d2b-8965-70f902a01c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:51.407412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed363d77-7e30-4d2b-8965-70f902a01c38 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.407692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed363d77-7e30-4d2b-8965-70f902a01c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.408082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed363d77-7e30-4d2b-8965-70f902a01c38 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.408808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed363d77-7e30-4d2b-8965-70f902a01c38 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:51.410402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed363d77-7e30-4d2b-8965-70f902a01c38 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:51.413860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:51.414531 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68adcf16-80b5-4951-a736-e59c6de89f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-68adcf16-80b5-4951-a736-e59c6de89f0e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:51.419570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:51.420538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:05:51.421396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-ed363d77-7e30-4d2b-8965-70f902a01c38 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:51.422049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:51.423727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:05:51.424791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:51.425759 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 451/896] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Aug 6 01:05:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:51.429366 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 446/897] 10.222.0.25 () {56 vars in 1132 bytes} [Sat Aug 6 01:05:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:51.431185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adcf16-80b5-4951-a736-e59c6de89f0e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:51.433698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed363d77-7e30-4d2b-8965-70f902a01c38 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:51.546939 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68adcf16-80b5-4951-a736-e59c6de89f0e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:51.549823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed363d77-7e30-4d2b-8965-70f902a01c38 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:51.564542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:51.586130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa28fc-6317-4450-9c88-247e3a504e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:51.586992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa28fc-6317-4450-9c88-247e3a504e97 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.588069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa28fc-6317-4450-9c88-247e3a504e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.588493 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa28fc-6317-4450-9c88-247e3a504e97 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.589647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5136ecc-12c4-415e-8cca-a379416c2f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:51.589968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa28fc-6317-4450-9c88-247e3a504e97 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:51.590584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa28fc-6317-4450-9c88-247e3a504e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:51.591136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5136ecc-12c4-415e-8cca-a379416c2f03 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.591493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5136ecc-12c4-415e-8cca-a379416c2f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.591858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5136ecc-12c4-415e-8cca-a379416c2f03 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.592616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5136ecc-12c4-415e-8cca-a379416c2f03 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:51.592987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5136ecc-12c4-415e-8cca-a379416c2f03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:51.593266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaa28fc-6317-4450-9c88-247e3a504e97 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:51.594470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5136ecc-12c4-415e-8cca-a379416c2f03 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:51.607711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 447/899] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:05:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:51.607711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5136ecc-12c4-415e-8cca-a379416c2f03 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:51.608272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 452/899] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Aug 6 01:05:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:51.609792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa28fc-6317-4450-9c88-247e3a504e97 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:51.675503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:51.718021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5136ecc-12c4-415e-8cca-a379416c2f03 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:51.718551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa28fc-6317-4450-9c88-247e3a504e97 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:51.754639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b89e2-c0bc-4ae0-9309-3e5c3cd762ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:51.755660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ea570-135c-4bfb-85d0-5dcfa7cf4fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:51.756004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b89e2-c0bc-4ae0-9309-3e5c3cd762ea None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.756308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ea570-135c-4bfb-85d0-5dcfa7cf4fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.756587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ea570-135c-4bfb-85d0-5dcfa7cf4fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.756890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ea570-135c-4bfb-85d0-5dcfa7cf4fc0 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.757171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b89e2-c0bc-4ae0-9309-3e5c3cd762ea None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.757572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ea570-135c-4bfb-85d0-5dcfa7cf4fc0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:51.757850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b89e2-c0bc-4ae0-9309-3e5c3cd762ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.757958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ea570-135c-4bfb-85d0-5dcfa7cf4fc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:51.759208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ea570-135c-4bfb-85d0-5dcfa7cf4fc0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:51.769259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 448/900] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:05:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:51.770172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ea570-135c-4bfb-85d0-5dcfa7cf4fc0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:51.771018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b89e2-c0bc-4ae0-9309-3e5c3cd762ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:51.773561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b89e2-c0bc-4ae0-9309-3e5c3cd762ea None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-2b3b89e2-c0bc-4ae0-9309-3e5c3cd762ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:51.789633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:51.790579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:51.791571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:51.791571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:51.791571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:51.791571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:51.798283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 453/901] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:51.799852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b89e2-c0bc-4ae0-9309-3e5c3cd762ea None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:51.831248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:51.879705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ea570-135c-4bfb-85d0-5dcfa7cf4fc0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:51.910017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:51.919277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260ed2d-83cf-4bdd-82a6-048e876064bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:51.919954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260ed2d-83cf-4bdd-82a6-048e876064bc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.920266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260ed2d-83cf-4bdd-82a6-048e876064bc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.920738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260ed2d-83cf-4bdd-82a6-048e876064bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.921417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260ed2d-83cf-4bdd-82a6-048e876064bc None admin] RBAC: Adding query filter params (interface=public, service_id=d49bee67e8194784b777d3ef5f3febe5, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:51.921733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260ed2d-83cf-4bdd-82a6-048e876064bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:51.923209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260ed2d-83cf-4bdd-82a6-048e876064bc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:51.930755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b89e2-c0bc-4ae0-9309-3e5c3cd762ea None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:51.933890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 449/902] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Aug 6 01:05:51 2022] GET /identity/v3/endpoints?service_id=d49bee67e8194784b777d3ef5f3febe5&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:51.934718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260ed2d-83cf-4bdd-82a6-048e876064bc None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:51.971105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccb84b-9f8e-4951-8145-84fc485b1e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:51.971753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccb84b-9f8e-4951-8145-84fc485b1e59 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:51.972288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccb84b-9f8e-4951-8145-84fc485b1e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:51.972581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccb84b-9f8e-4951-8145-84fc485b1e59 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:51.974067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccb84b-9f8e-4951-8145-84fc485b1e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:51.974375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccb84b-9f8e-4951-8145-84fc485b1e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:51.976399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ccb84b-9f8e-4951-8145-84fc485b1e59 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:51.988991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 454/903] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:51.994387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccb84b-9f8e-4951-8145-84fc485b1e59 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:52.057808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260ed2d-83cf-4bdd-82a6-048e876064bc None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:52.088965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:52.094994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0886a5f-f6b6-4b80-896f-d4fc88d3232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:52.095610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886a5f-f6b6-4b80-896f-d4fc88d3232f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:52.095925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886a5f-f6b6-4b80-896f-d4fc88d3232f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:52.096193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886a5f-f6b6-4b80-896f-d4fc88d3232f None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:52.096841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0886a5f-f6b6-4b80-896f-d4fc88d3232f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:52.098197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0886a5f-f6b6-4b80-896f-d4fc88d3232f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:52.098411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccb84b-9f8e-4951-8145-84fc485b1e59 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:52.107977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 450/904] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Aug 6 01:05:51 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:05:52.110704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eafc81-9cf9-49b0-8c8e-748998a6abba None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:52.110992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eafc81-9cf9-49b0-8c8e-748998a6abba None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:52.111266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eafc81-9cf9-49b0-8c8e-748998a6abba None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:52.113049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 451/905] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:52.121631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4e988-dfe6-43c5-9927-0b526a5f74d5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:52.122060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4e988-dfe6-43c5-9927-0b526a5f74d5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:52.122459 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4e988-dfe6-43c5-9927-0b526a5f74d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:52.142297 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712a362-1267-47af-9167-b7ea57d4d20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:52.143265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712a362-1267-47af-9167-b7ea57d4d20c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:52.143696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712a362-1267-47af-9167-b7ea57d4d20c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:52.144338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712a362-1267-47af-9167-b7ea57d4d20c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:52.146379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-63e4e988-dfe6-43c5-9927-0b526a5f74d5 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:52.149205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-63e4e988-dfe6-43c5-9927-0b526a5f74d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:52.155264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712a362-1267-47af-9167-b7ea57d4d20c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:52.157124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712a362-1267-47af-9167-b7ea57d4d20c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-a712a362-1267-47af-9167-b7ea57d4d20c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:52.175091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:52.176192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:52.177163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:52.177163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:52.177163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:52.177163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:52.177163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:52.177163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:52.177163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:52.177163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:52.177163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:05:52.177163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:52.179903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 455/906] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Aug 6 01:05:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:52.186752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712a362-1267-47af-9167-b7ea57d4d20c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:52.219543 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:52.238189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 452/907] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:52.260555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4e988-dfe6-43c5-9927-0b526a5f74d5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:52.327224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712a362-1267-47af-9167-b7ea57d4d20c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:52.371249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345ccbc-01c3-4862-8d78-934efedfd5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:52.372428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345ccbc-01c3-4862-8d78-934efedfd5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:52.373110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345ccbc-01c3-4862-8d78-934efedfd5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:52.373533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345ccbc-01c3-4862-8d78-934efedfd5b9 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:52.375155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345ccbc-01c3-4862-8d78-934efedfd5b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:52.375640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345ccbc-01c3-4862-8d78-934efedfd5b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:52.378102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345ccbc-01c3-4862-8d78-934efedfd5b9 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:52.379487 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4e988-dfe6-43c5-9927-0b526a5f74d5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:52.404214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 456/908] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Aug 6 01:05:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:52.405841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345ccbc-01c3-4862-8d78-934efedfd5b9 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:52.417109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:52.429963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9098ac4-313d-4386-b712-5250a6f98427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:52.430937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9098ac4-313d-4386-b712-5250a6f98427 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:52.431385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9098ac4-313d-4386-b712-5250a6f98427 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:52.431861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9098ac4-313d-4386-b712-5250a6f98427 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:52.433706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9098ac4-313d-4386-b712-5250a6f98427 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:52.435184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9098ac4-313d-4386-b712-5250a6f98427 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-c9098ac4-313d-4386-b712-5250a6f98427 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:52.448726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:52.450859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:52.453453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 453/909] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:52.463810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9098ac4-313d-4386-b712-5250a6f98427 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:52.530865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345ccbc-01c3-4862-8d78-934efedfd5b9 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:52.532243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:52.577255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1d5bd-eb79-441b-96b3-3d63d76b3299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:52.578478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1d5bd-eb79-441b-96b3-3d63d76b3299 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:52.578975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1d5bd-eb79-441b-96b3-3d63d76b3299 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:52.579428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1d5bd-eb79-441b-96b3-3d63d76b3299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:52.593951 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9098ac4-313d-4386-b712-5250a6f98427 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:52.640741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee1798-f1a9-4ea3-900f-3a60ddaf0b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:52.641479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee1798-f1a9-4ea3-900f-3a60ddaf0b01 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:52.642077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee1798-f1a9-4ea3-900f-3a60ddaf0b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:52.642425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee1798-f1a9-4ea3-900f-3a60ddaf0b01 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:52.643184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee1798-f1a9-4ea3-900f-3a60ddaf0b01 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:52.643836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee1798-f1a9-4ea3-900f-3a60ddaf0b01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:52.645962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee1798-f1a9-4ea3-900f-3a60ddaf0b01 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:52.655812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 454/910] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:52.655968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1d5bd-eb79-441b-96b3-3d63d76b3299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:52.657524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1d5bd-eb79-441b-96b3-3d63d76b3299 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:52.658350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee1798-f1a9-4ea3-900f-3a60ddaf0b01 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:52.669905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:52.776273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee1798-f1a9-4ea3-900f-3a60ddaf0b01 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:52.781273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 457/911] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:05:52 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 01:05:52.787581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1d5bd-eb79-441b-96b3-3d63d76b3299 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:52.795600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:52.817186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e80771-572d-40e5-aa3a-87b2bd072dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:52.818218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e80771-572d-40e5-aa3a-87b2bd072dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:52.818713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e80771-572d-40e5-aa3a-87b2bd072dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:52.819187 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e80771-572d-40e5-aa3a-87b2bd072dc2 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:52.820154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e80771-572d-40e5-aa3a-87b2bd072dc2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:52.820681 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e80771-572d-40e5-aa3a-87b2bd072dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:52.822250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e80771-572d-40e5-aa3a-87b2bd072dc2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:52.832193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 455/912] 10.222.0.25 () {56 vars in 1114 bytes} [Sat Aug 6 01:05:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:52.834045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e80771-572d-40e5-aa3a-87b2bd072dc2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:52.904404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1d5bd-eb79-441b-96b3-3d63d76b3299 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:52.917684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:52.950913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9e242-4498-4f26-a19e-3b0bbdaf7b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:52.951868 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9e242-4498-4f26-a19e-3b0bbdaf7b6c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:52.952157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9e242-4498-4f26-a19e-3b0bbdaf7b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:52.952433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9e242-4498-4f26-a19e-3b0bbdaf7b6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:52.963572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e80771-572d-40e5-aa3a-87b2bd072dc2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:52.967249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9e242-4498-4f26-a19e-3b0bbdaf7b6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:52.968763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9e242-4498-4f26-a19e-3b0bbdaf7b6c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-30a9e242-4498-4f26-a19e-3b0bbdaf7b6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:52.988366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:52.989296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:52.990201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:52.990201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:52.990201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:52.990201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:52.994893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 458/913] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:52.996608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9e242-4498-4f26-a19e-3b0bbdaf7b6c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:53.008097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9027bf-079e-4ad2-b651-1c19ff93d7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:53.009031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9027bf-079e-4ad2-b651-1c19ff93d7c3 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:53.009518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9027bf-079e-4ad2-b651-1c19ff93d7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:53.010104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9027bf-079e-4ad2-b651-1c19ff93d7c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:53.011022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9027bf-079e-4ad2-b651-1c19ff93d7c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:53.012697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9027bf-079e-4ad2-b651-1c19ff93d7c3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:53.103454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 456/914] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:05:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:05:53.117863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9e242-4498-4f26-a19e-3b0bbdaf7b6c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:53.162711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575e120-b83f-4d65-802a-90e2404996e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:53.164277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575e120-b83f-4d65-802a-90e2404996e1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:53.164768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575e120-b83f-4d65-802a-90e2404996e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:53.165245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575e120-b83f-4d65-802a-90e2404996e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:53.166265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575e120-b83f-4d65-802a-90e2404996e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:53.166832 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575e120-b83f-4d65-802a-90e2404996e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:53.168678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575e120-b83f-4d65-802a-90e2404996e1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:53.183820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 459/915] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:53.192457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9027bf-079e-4ad2-b651-1c19ff93d7c3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:53.213273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1892a3-178f-495d-966e-e4aab7ac3aa4 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:53.213838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1892a3-178f-495d-966e-e4aab7ac3aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:53.214124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1892a3-178f-495d-966e-e4aab7ac3aa4 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:53.215628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 460/916] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:53.221000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d362c1-6036-4c50-b589-6602a6fe9b28 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:53.221293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d362c1-6036-4c50-b589-6602a6fe9b28 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:53.221549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d362c1-6036-4c50-b589-6602a6fe9b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:53.242062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e4d362c1-6036-4c50-b589-6602a6fe9b28 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:53.243908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e4d362c1-6036-4c50-b589-6602a6fe9b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:53.259625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:53.276174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:53.314439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9027bf-079e-4ad2-b651-1c19ff93d7c3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:53.341187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 461/917] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:05:53.361664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62561cae-bb65-493c-9cd0-6fb68cf938c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:53.362381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62561cae-bb65-493c-9cd0-6fb68cf938c5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:53.362672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62561cae-bb65-493c-9cd0-6fb68cf938c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:53.362950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62561cae-bb65-493c-9cd0-6fb68cf938c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:53.364080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d362c1-6036-4c50-b589-6602a6fe9b28 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:53.371347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62561cae-bb65-493c-9cd0-6fb68cf938c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:53.372851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62561cae-bb65-493c-9cd0-6fb68cf938c5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-62561cae-bb65-493c-9cd0-6fb68cf938c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:53.387185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:53.388194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:53.389105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:53.389105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:53.389105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:53.389105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:53.389105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:53.389105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:53.389105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:53.389105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:53.391914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 457/918] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:53.395446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62561cae-bb65-493c-9cd0-6fb68cf938c5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:53.501943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d362c1-6036-4c50-b589-6602a6fe9b28 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:53.508426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:53.515245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62561cae-bb65-493c-9cd0-6fb68cf938c5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:53.553306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc6a44-29fa-454d-8732-989cde3e3e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:53.553986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc6a44-29fa-454d-8732-989cde3e3e58 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:53.554264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc6a44-29fa-454d-8732-989cde3e3e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:53.554530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc6a44-29fa-454d-8732-989cde3e3e58 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:53.555154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc6a44-29fa-454d-8732-989cde3e3e58 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:53.556548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc6a44-29fa-454d-8732-989cde3e3e58 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:53.560526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29ef9d-2788-49ec-97ff-42f3db1be520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:53.561400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29ef9d-2788-49ec-97ff-42f3db1be520 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:53.561883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29ef9d-2788-49ec-97ff-42f3db1be520 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:53.562357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29ef9d-2788-49ec-97ff-42f3db1be520 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:53.563385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29ef9d-2788-49ec-97ff-42f3db1be520 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:53.563938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29ef9d-2788-49ec-97ff-42f3db1be520 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:53.565555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29ef9d-2788-49ec-97ff-42f3db1be520 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d9cc6a44-29fa-454d-8732-989cde3e3e58 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:53.565681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:53.566636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:53.567503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:53.572028 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 462/919] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:05:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:53.573325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc6a44-29fa-454d-8732-989cde3e3e58 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:53.591660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 458/920] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:53.593427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29ef9d-2788-49ec-97ff-42f3db1be520 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:53.595407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:53.684912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc6a44-29fa-454d-8732-989cde3e3e58 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:53.706291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29ef9d-2788-49ec-97ff-42f3db1be520 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:53.720231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d63b9-b110-44fb-b74d-fb0acc5a93e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:53.720799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d63b9-b110-44fb-b74d-fb0acc5a93e8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:53.721080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d63b9-b110-44fb-b74d-fb0acc5a93e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:53.721354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d63b9-b110-44fb-b74d-fb0acc5a93e8 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:53.722031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d63b9-b110-44fb-b74d-fb0acc5a93e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:53.722335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d63b9-b110-44fb-b74d-fb0acc5a93e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:53.723709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d63b9-b110-44fb-b74d-fb0acc5a93e8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:53.733545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 463/921] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Aug 6 01:05:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:53.734536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d63b9-b110-44fb-b74d-fb0acc5a93e8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:53.746911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefa36c-93a3-474b-8cec-e4dc519878c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:53.749691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefa36c-93a3-474b-8cec-e4dc519878c3 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:53.750134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefa36c-93a3-474b-8cec-e4dc519878c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:53.750559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefa36c-93a3-474b-8cec-e4dc519878c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:53.751504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefa36c-93a3-474b-8cec-e4dc519878c3 None admin] RBAC: Adding query filter params (role.id=c4fac6ed74ff415cb3e604007bcf0161, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=6d62ec1463c14665b94e9b2d85e48e14) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:53.752012 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefa36c-93a3-474b-8cec-e4dc519878c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:53.753788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefa36c-93a3-474b-8cec-e4dc519878c3 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:53.771983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 459/922] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:05:53 2022] GET /identity/v3/role_assignments?user.id=6d62ec1463c14665b94e9b2d85e48e14&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=c4fac6ed74ff415cb3e604007bcf0161 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:53.818610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:53.847010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d63b9-b110-44fb-b74d-fb0acc5a93e8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:53.885614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f29454-8482-4219-a10e-455f824279d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:53.886253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f29454-8482-4219-a10e-455f824279d7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:53.890663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f29454-8482-4219-a10e-455f824279d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:53.890663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f29454-8482-4219-a10e-455f824279d7 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:53.890663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f29454-8482-4219-a10e-455f824279d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:53.890663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f29454-8482-4219-a10e-455f824279d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:53.890663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f29454-8482-4219-a10e-455f824279d7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:53.895901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 464/923] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Aug 6 01:05:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:53.902586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefa36c-93a3-474b-8cec-e4dc519878c3 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:53.924764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:54.012600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefa36c-93a3-474b-8cec-e4dc519878c3 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:54.051707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957337a-1800-4b15-b70e-cfd6896175ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:54.052358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957337a-1800-4b15-b70e-cfd6896175ac None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:54.052644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957337a-1800-4b15-b70e-cfd6896175ac None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:54.052922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957337a-1800-4b15-b70e-cfd6896175ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:54.053686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957337a-1800-4b15-b70e-cfd6896175ac None admin] RBAC: Adding query filter params (interface=public, service_id=b8f1e6f016304f5e8c29cdec46bc715a, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:54.054568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957337a-1800-4b15-b70e-cfd6896175ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:54.055926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957337a-1800-4b15-b70e-cfd6896175ac None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:54.061811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 460/924] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Aug 6 01:05:53 2022] GET /identity/v3/endpoints?service_id=b8f1e6f016304f5e8c29cdec46bc715a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:54.067150 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f29454-8482-4219-a10e-455f824279d7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:54.177027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f29454-8482-4219-a10e-455f824279d7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:54.201626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ae2d3-41db-489b-9429-a59857e50d04 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:54.201909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ae2d3-41db-489b-9429-a59857e50d04 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:54.202524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ae2d3-41db-489b-9429-a59857e50d04 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:54.204800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 461/925] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:54.209523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f928d4b-9e3d-454a-bd14-dc49370b802b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:54.209808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f928d4b-9e3d-454a-bd14-dc49370b802b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:54.210153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f928d4b-9e3d-454a-bd14-dc49370b802b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:54.216279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112962a-d010-4a4f-89f0-089211dc1d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:54.217123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112962a-d010-4a4f-89f0-089211dc1d5a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:54.217580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112962a-d010-4a4f-89f0-089211dc1d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:54.217853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112962a-d010-4a4f-89f0-089211dc1d5a None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:54.218509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112962a-d010-4a4f-89f0-089211dc1d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:54.219967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112962a-d010-4a4f-89f0-089211dc1d5a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:54.226260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 465/926] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Aug 6 01:05:54 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:05:54.228975 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6f928d4b-9e3d-454a-bd14-dc49370b802b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:54.230345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6f928d4b-9e3d-454a-bd14-dc49370b802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:54.322412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 462/927] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:05:54.346022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112962a-d010-4a4f-89f0-089211dc1d5a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:54.349480 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:54.455229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112962a-d010-4a4f-89f0-089211dc1d5a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:54.489731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053d40a-a66d-4a55-9a96-47c8ac4ae77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:54.490374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053d40a-a66d-4a55-9a96-47c8ac4ae77d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:54.490654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053d40a-a66d-4a55-9a96-47c8ac4ae77d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:54.490920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053d40a-a66d-4a55-9a96-47c8ac4ae77d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:54.492274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053d40a-a66d-4a55-9a96-47c8ac4ae77d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:54.493701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4053d40a-a66d-4a55-9a96-47c8ac4ae77d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4053d40a-a66d-4a55-9a96-47c8ac4ae77d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:54.503382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:54.504540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:54.505563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 466/928] 10.222.0.25 () {56 vars in 1132 bytes} [Sat Aug 6 01:05:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:54.510864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f928d4b-9e3d-454a-bd14-dc49370b802b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:54.525567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:54.630395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f928d4b-9e3d-454a-bd14-dc49370b802b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:54.665942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414f3f3-3962-4799-86e8-0576aaa00080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:54.666566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414f3f3-3962-4799-86e8-0576aaa00080 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:54.666850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414f3f3-3962-4799-86e8-0576aaa00080 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:54.667129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414f3f3-3962-4799-86e8-0576aaa00080 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:54.667853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414f3f3-3962-4799-86e8-0576aaa00080 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:54.668169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414f3f3-3962-4799-86e8-0576aaa00080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:54.669534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414f3f3-3962-4799-86e8-0576aaa00080 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:54.674743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 463/929] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:05:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:54.680902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053d40a-a66d-4a55-9a96-47c8ac4ae77d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:54.748696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:54.783755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053d40a-a66d-4a55-9a96-47c8ac4ae77d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:54.818685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352d554-e8ba-4613-b946-0e078f7f2529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:54.819299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352d554-e8ba-4613-b946-0e078f7f2529 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:54.819617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352d554-e8ba-4613-b946-0e078f7f2529 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:54.819929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352d554-e8ba-4613-b946-0e078f7f2529 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:54.820612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352d554-e8ba-4613-b946-0e078f7f2529 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:54.820920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352d554-e8ba-4613-b946-0e078f7f2529 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:54.822305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352d554-e8ba-4613-b946-0e078f7f2529 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:54.827653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 467/930] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:05:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:54.833665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414f3f3-3962-4799-86e8-0576aaa00080 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:54.916169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:54.939727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414f3f3-3962-4799-86e8-0576aaa00080 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:54.976200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf664e-36ef-4aef-b0a5-a37d7cee5bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:54.976845 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf664e-36ef-4aef-b0a5-a37d7cee5bee None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:54.977129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf664e-36ef-4aef-b0a5-a37d7cee5bee None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:54.977405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf664e-36ef-4aef-b0a5-a37d7cee5bee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:54.978143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf664e-36ef-4aef-b0a5-a37d7cee5bee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:54.979507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf664e-36ef-4aef-b0a5-a37d7cee5bee None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:55.064289 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 464/931] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:05:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:05:55.262355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46229ae1-7e11-4a21-9ab0-0ba0e023cbb9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:55.262827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46229ae1-7e11-4a21-9ab0-0ba0e023cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:55.262974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46229ae1-7e11-4a21-9ab0-0ba0e023cbb9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:55.264583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 468/932] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:55.269804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d66c6e-f2e9-40b4-ad9d-46420ff3bb29 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:55.270098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d66c6e-f2e9-40b4-ad9d-46420ff3bb29 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:55.270380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d66c6e-f2e9-40b4-ad9d-46420ff3bb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:55.289215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-80d66c6e-f2e9-40b4-ad9d-46420ff3bb29 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:55.290413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-80d66c6e-f2e9-40b4-ad9d-46420ff3bb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:55.365830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 465/933] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:55.389028 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46229ae1-7e11-4a21-9ab0-0ba0e023cbb9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:55.495647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46229ae1-7e11-4a21-9ab0-0ba0e023cbb9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:55.528839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:55.531129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:55.531758 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:55.532061 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:55.532335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:55.533032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:55.534461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:55.537153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WARNING keystone.server.flask.common [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 06 01:05:55.593719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.sql.core [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 01:05:55.593719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 06 01:05:55.593719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [parameters: {'id': 'd0a401ec9c5d474388d0e12a82e8f42e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 06 01:05:55.593719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80524) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d0a401ec9c5d474388d0e12a82e8f42e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d0a401ec9c5d474388d0e12a82e8f42e. Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:05:55.609714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result.read() Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 06 01:05:55.610606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application yield resource Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise self.value Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application session.commit() Aug 06 01:05:55.611609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.session.flush() Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application compat.raise_( Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 01:05:55.612565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 01:05:55.614385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result.read() Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:05:55.617065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [parameters: {'id': 'd0a401ec9c5d474388d0e12a82e8f42e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:55.619242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:55.621186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:55.625011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:55.625011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 06 01:05:55.625011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 06 01:05:55.625011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:55.625011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:55.625011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 06 01:05:55.625011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 06 01:05:55.625011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d0a401ec9c5d474388d0e12a82e8f42e. Aug 06 01:05:55.625011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:55.625011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 469/934] 10.222.0.25 () {60 vars in 1140 bytes} [Sat Aug 6 01:05:55 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:55.626433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d66c6e-f2e9-40b4-ad9d-46420ff3bb29 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:55.700217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:55.736730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d66c6e-f2e9-40b4-ad9d-46420ff3bb29 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:55.774090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb493b3c-716f-42bb-b341-90a1bc322e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:55.775123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb493b3c-716f-42bb-b341-90a1bc322e72 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:55.775626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb493b3c-716f-42bb-b341-90a1bc322e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:55.776339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb493b3c-716f-42bb-b341-90a1bc322e72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:55.783511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb493b3c-716f-42bb-b341-90a1bc322e72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:55.785206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb493b3c-716f-42bb-b341-90a1bc322e72 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-eb493b3c-716f-42bb-b341-90a1bc322e72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:55.797527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:55.798497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:55.799402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:55.799402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:55.799402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:55.799402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:55.799402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:55.799402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:55.799402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:55.799402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:55.803326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 466/935] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:55.809674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:55.925599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e691c9b2-3d83-45a8-8db1-b99d840514d8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:55.964610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac5a47-b92d-4cfb-8164-71fe4293502c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:55.965602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac5a47-b92d-4cfb-8164-71fe4293502c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:55.966090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac5a47-b92d-4cfb-8164-71fe4293502c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:55.966595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac5a47-b92d-4cfb-8164-71fe4293502c None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:55.967544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac5a47-b92d-4cfb-8164-71fe4293502c None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:55.968088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac5a47-b92d-4cfb-8164-71fe4293502c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:55.970018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac5a47-b92d-4cfb-8164-71fe4293502c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:55.992195 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 470/936] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:05:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:56.042612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3eed80-24d1-403a-9307-4b1563cc7ce6 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:56.043109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3eed80-24d1-403a-9307-4b1563cc7ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:56.043109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3eed80-24d1-403a-9307-4b1563cc7ce6 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:56.044955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 467/937] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:56.050883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99587c38-31e0-4769-8982-5448308d3152 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:56.051184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99587c38-31e0-4769-8982-5448308d3152 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:56.051450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99587c38-31e0-4769-8982-5448308d3152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:56.075395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-99587c38-31e0-4769-8982-5448308d3152 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:56.077364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-99587c38-31e0-4769-8982-5448308d3152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:56.128129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:56.154718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 471/938] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:56.179688 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3eed80-24d1-403a-9307-4b1563cc7ce6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:56.298129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3eed80-24d1-403a-9307-4b1563cc7ce6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:56.317160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:56.334717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd00d9-9832-4031-bdba-e916af960bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:56.340574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd00d9-9832-4031-bdba-e916af960bbb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:56.340574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd00d9-9832-4031-bdba-e916af960bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:56.340574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd00d9-9832-4031-bdba-e916af960bbb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:56.347203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd00d9-9832-4031-bdba-e916af960bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-61dd00d9-9832-4031-bdba-e916af960bbb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:05:56.353351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:56.354301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:56.355151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 468/939] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Aug 6 01:05:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:56.360956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99587c38-31e0-4769-8982-5448308d3152 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:56.460610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93365d22-4bec-4dea-aca4-01bb539fe024 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:56.460957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93365d22-4bec-4dea-aca4-01bb539fe024 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:56.461189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93365d22-4bec-4dea-aca4-01bb539fe024 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:56.463712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 469/940] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:56.465015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99587c38-31e0-4769-8982-5448308d3152 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:56.468518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d31a6-f512-4502-9b4e-c71d59376410 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:56.468518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d31a6-f512-4502-9b4e-c71d59376410 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:56.468518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d31a6-f512-4502-9b4e-c71d59376410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:56.488545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ee5d31a6-f512-4502-9b4e-c71d59376410 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:56.489829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ee5d31a6-f512-4502-9b4e-c71d59376410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:56.509532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e97d6-5d9e-457a-8f2c-df03bc8859d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:56.510372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e97d6-5d9e-457a-8f2c-df03bc8859d6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:56.510775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e97d6-5d9e-457a-8f2c-df03bc8859d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:56.511053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e97d6-5d9e-457a-8f2c-df03bc8859d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:56.511936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e97d6-5d9e-457a-8f2c-df03bc8859d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:56.512243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e97d6-5d9e-457a-8f2c-df03bc8859d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:56.513718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e97d6-5d9e-457a-8f2c-df03bc8859d6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:56.522901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 472/941] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Aug 6 01:05:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:56.532947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e97d6-5d9e-457a-8f2c-df03bc8859d6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:56.585609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:56.603773 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 470/942] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:05:56.631696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d31a6-f512-4502-9b4e-c71d59376410 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:56.661525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e97d6-5d9e-457a-8f2c-df03bc8859d6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:56.709129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1aeae8-1a61-4c13-9a3b-55333bbbf3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:56.711076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aeae8-1a61-4c13-9a3b-55333bbbf3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:56.711552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aeae8-1a61-4c13-9a3b-55333bbbf3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:56.712039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aeae8-1a61-4c13-9a3b-55333bbbf3c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:56.726741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1aeae8-1a61-4c13-9a3b-55333bbbf3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:56.729556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1aeae8-1a61-4c13-9a3b-55333bbbf3c0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:56.744641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d31a6-f512-4502-9b4e-c71d59376410 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ed1aeae8-1a61-4c13-9a3b-55333bbbf3c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:56.751403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:56.752327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:56.753263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:56.753263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:56.753263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:56.753263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:56.760227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 473/943] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:56.762199 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:56.762903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1aeae8-1a61-4c13-9a3b-55333bbbf3c0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:56.788942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f3a4e-f507-499e-a9e8-46801154006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:56.789769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f3a4e-f507-499e-a9e8-46801154006d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:56.790138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f3a4e-f507-499e-a9e8-46801154006d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:56.790510 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f3a4e-f507-499e-a9e8-46801154006d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:56.791238 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f3a4e-f507-499e-a9e8-46801154006d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:56.792991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f3a4e-f507-499e-a9e8-46801154006d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-657f3a4e-f507-499e-a9e8-46801154006d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:56.804432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:56.805668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:56.806729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:56.813523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 471/944] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:05:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:05:56.814891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f3a4e-f507-499e-a9e8-46801154006d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:56.881977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:56.892913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1aeae8-1a61-4c13-9a3b-55333bbbf3c0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:56.932825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6648559-7ca1-49f1-8473-15931dba90a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:56.934217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6648559-7ca1-49f1-8473-15931dba90a0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:56.934697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6648559-7ca1-49f1-8473-15931dba90a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:56.935185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6648559-7ca1-49f1-8473-15931dba90a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:56.936415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6648559-7ca1-49f1-8473-15931dba90a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:56.937148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6648559-7ca1-49f1-8473-15931dba90a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:56.939747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6648559-7ca1-49f1-8473-15931dba90a0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:56.947594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f3a4e-f507-499e-a9e8-46801154006d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:56.952392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 474/945] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:56.958620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6648559-7ca1-49f1-8473-15931dba90a0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:56.988607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4feafea-1197-4632-bec3-09b7925be843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:56.989558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4feafea-1197-4632-bec3-09b7925be843 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:56.990161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4feafea-1197-4632-bec3-09b7925be843 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:56.990720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4feafea-1197-4632-bec3-09b7925be843 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:56.991661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4feafea-1197-4632-bec3-09b7925be843 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:56.992188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4feafea-1197-4632-bec3-09b7925be843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:56.994069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4feafea-1197-4632-bec3-09b7925be843 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:57.003985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 472/946] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Aug 6 01:05:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:57.005682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4feafea-1197-4632-bec3-09b7925be843 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:57.046654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:57.067192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6648559-7ca1-49f1-8473-15931dba90a0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:57.105296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a0607-e11d-4108-a067-7bb9db78473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:57.106294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a0607-e11d-4108-a067-7bb9db78473b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.106728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a0607-e11d-4108-a067-7bb9db78473b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.107187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a0607-e11d-4108-a067-7bb9db78473b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.115808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a0607-e11d-4108-a067-7bb9db78473b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:57.117505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a0607-e11d-4108-a067-7bb9db78473b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:57.118488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4feafea-1197-4632-bec3-09b7925be843 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e92a0607-e11d-4108-a067-7bb9db78473b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:57.129871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:57.130842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:57.131863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:57.131863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:57.131863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:57.131863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:57.131863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:57.131863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:57.131863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:57.131863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:57.131863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:05:57.131863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:57.139152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 475/947] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Aug 6 01:05:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:57.140214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a0607-e11d-4108-a067-7bb9db78473b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:57.155465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:57.158334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787b758-c312-4faf-9b43-cd8239714f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:57.159015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787b758-c312-4faf-9b43-cd8239714f2d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.159304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787b758-c312-4faf-9b43-cd8239714f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.159578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787b758-c312-4faf-9b43-cd8239714f2d None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.160297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787b758-c312-4faf-9b43-cd8239714f2d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:57.160602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787b758-c312-4faf-9b43-cd8239714f2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:57.161986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787b758-c312-4faf-9b43-cd8239714f2d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:57.167825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 473/948] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Aug 6 01:05:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:57.169049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b66f91-9954-4f9b-8eb5-e70e18675270 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.169546 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b66f91-9954-4f9b-8eb5-e70e18675270 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.169824 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b66f91-9954-4f9b-8eb5-e70e18675270 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.171173 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 474/949] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:57.173337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b66f91-9954-4f9b-8eb5-e70e18675270 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:57.252509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a0607-e11d-4108-a067-7bb9db78473b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:57.280821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:57.285237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b66f91-9954-4f9b-8eb5-e70e18675270 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:57.285750 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403521b6-2aaa-40ee-b62b-5fb2e04f0d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:57.286376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403521b6-2aaa-40ee-b62b-5fb2e04f0d9b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.286662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403521b6-2aaa-40ee-b62b-5fb2e04f0d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.286918 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403521b6-2aaa-40ee-b62b-5fb2e04f0d9b None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.288114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403521b6-2aaa-40ee-b62b-5fb2e04f0d9b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:57.288114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403521b6-2aaa-40ee-b62b-5fb2e04f0d9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:57.290190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403521b6-2aaa-40ee-b62b-5fb2e04f0d9b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:57.311809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 476/950] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Aug 6 01:05:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:57.311809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a267d0d-98ac-4013-8f87-1483ae936af9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.311809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a267d0d-98ac-4013-8f87-1483ae936af9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.311809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a267d0d-98ac-4013-8f87-1483ae936af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.326984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec47d9-7fb1-4c2c-81ac-0c62287a5ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:57.327605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec47d9-7fb1-4c2c-81ac-0c62287a5ec8 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.327709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9a267d0d-98ac-4013-8f87-1483ae936af9 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:57.327935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec47d9-7fb1-4c2c-81ac-0c62287a5ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.328213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec47d9-7fb1-4c2c-81ac-0c62287a5ec8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.329090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9a267d0d-98ac-4013-8f87-1483ae936af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:57.329198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec47d9-7fb1-4c2c-81ac-0c62287a5ec8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:57.330522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec47d9-7fb1-4c2c-81ac-0c62287a5ec8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:57.388362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:57.393840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 477/951] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:57.394568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a267d0d-98ac-4013-8f87-1483ae936af9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:57.431842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 475/952] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:05:57 2022] POST /identity/v3/endpoints => generated 349 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:05:57.431842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec47d9-7fb1-4c2c-81ac-0c62287a5ec8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:57.517167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a267d0d-98ac-4013-8f87-1483ae936af9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:57.563752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808811fa-2674-4298-8e4e-fddcf29e8475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:57.564469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808811fa-2674-4298-8e4e-fddcf29e8475 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.564753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808811fa-2674-4298-8e4e-fddcf29e8475 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.565030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808811fa-2674-4298-8e4e-fddcf29e8475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.577897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808811fa-2674-4298-8e4e-fddcf29e8475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:57.579608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808811fa-2674-4298-8e4e-fddcf29e8475 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:57.585804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec47d9-7fb1-4c2c-81ac-0c62287a5ec8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-808811fa-2674-4298-8e4e-fddcf29e8475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:05:57.599641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:57.600615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:57.601533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:05:57.601533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:05:57.601533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:05:57.601533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:57.602160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 478/953] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:57.607740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808811fa-2674-4298-8e4e-fddcf29e8475 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:57.630142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2e9e7-2027-4670-b73b-88d704a7a472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:57.631090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2e9e7-2027-4670-b73b-88d704a7a472 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.631376 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2e9e7-2027-4670-b73b-88d704a7a472 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.631649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2e9e7-2027-4670-b73b-88d704a7a472 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.632883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2e9e7-2027-4670-b73b-88d704a7a472 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:57.634675 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2e9e7-2027-4670-b73b-88d704a7a472 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-11e2e9e7-2027-4670-b73b-88d704a7a472 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:57.643980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:57.645090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:57.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 476/954] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:57.653117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2e9e7-2027-4670-b73b-88d704a7a472 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:57.671887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:57.677398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:57.745332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808811fa-2674-4298-8e4e-fddcf29e8475 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:57.764012 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2e9e7-2027-4670-b73b-88d704a7a472 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:57.788999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b24d3-5364-49b6-9617-5550f6c4a2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:57.789729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b24d3-5364-49b6-9617-5550f6c4a2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.790078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b24d3-5364-49b6-9617-5550f6c4a2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.790460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b24d3-5364-49b6-9617-5550f6c4a2b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.791217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b24d3-5364-49b6-9617-5550f6c4a2b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:57.791610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b24d3-5364-49b6-9617-5550f6c4a2b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:57.793517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b24d3-5364-49b6-9617-5550f6c4a2b4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:57.811827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 479/955] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:57.812678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67708512-ff34-41d7-89e2-d19d3d28aa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:57.813471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67708512-ff34-41d7-89e2-d19d3d28aa6e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.813943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67708512-ff34-41d7-89e2-d19d3d28aa6e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.814611 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67708512-ff34-41d7-89e2-d19d3d28aa6e None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.815502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b24d3-5364-49b6-9617-5550f6c4a2b4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:57.815620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67708512-ff34-41d7-89e2-d19d3d28aa6e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:57.816090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67708512-ff34-41d7-89e2-d19d3d28aa6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:57.817463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67708512-ff34-41d7-89e2-d19d3d28aa6e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:57.824817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 477/956] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:05:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:57.830309 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67708512-ff34-41d7-89e2-d19d3d28aa6e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:57.926463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:57.943915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67708512-ff34-41d7-89e2-d19d3d28aa6e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:57.958058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b24d3-5364-49b6-9617-5550f6c4a2b4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:57.968947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:57.982839 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654c7ac-ee33-4116-89b5-0de8059255b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:57.983681 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654c7ac-ee33-4116-89b5-0de8059255b5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:57.984165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654c7ac-ee33-4116-89b5-0de8059255b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:57.984606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654c7ac-ee33-4116-89b5-0de8059255b5 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:57.985421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654c7ac-ee33-4116-89b5-0de8059255b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:57.985919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654c7ac-ee33-4116-89b5-0de8059255b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:57.987455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7654c7ac-ee33-4116-89b5-0de8059255b5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:58.000048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 478/957] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:05:57 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:05:58.000917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f0ae8-9367-476b-9b12-49d4a2eaa2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:58.001629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f0ae8-9367-476b-9b12-49d4a2eaa2df None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:58.001937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f0ae8-9367-476b-9b12-49d4a2eaa2df None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:58.002206 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f0ae8-9367-476b-9b12-49d4a2eaa2df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:58.010630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f0ae8-9367-476b-9b12-49d4a2eaa2df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:58.012186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f0ae8-9367-476b-9b12-49d4a2eaa2df None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-f54f0ae8-9367-476b-9b12-49d4a2eaa2df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:05:58.025243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:58.026214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:58.027197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:05:58.027197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:05:58.027197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:05:58.027197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:05:58.027197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:05:58.027197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:05:58.027197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:05:58.027197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:58.027197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 480/958] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:58.032712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654c7ac-ee33-4116-89b5-0de8059255b5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:58.145786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654c7ac-ee33-4116-89b5-0de8059255b5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:58.183054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa84c4-d5fc-455f-ac89-1bf716bde583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:58.183605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa84c4-d5fc-455f-ac89-1bf716bde583 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:58.183926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa84c4-d5fc-455f-ac89-1bf716bde583 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:58.184201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa84c4-d5fc-455f-ac89-1bf716bde583 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:58.184983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa84c4-d5fc-455f-ac89-1bf716bde583 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:58.185282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa84c4-d5fc-455f-ac89-1bf716bde583 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:58.187550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa84c4-d5fc-455f-ac89-1bf716bde583 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:58.188128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392f8a5-5f89-4ec2-babe-989c4e95b1a0 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:58.188416 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392f8a5-5f89-4ec2-babe-989c4e95b1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:58.188695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392f8a5-5f89-4ec2-babe-989c4e95b1a0 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:58.190298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 481/959] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:05:58.199192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bd6d1-bcfc-4376-9a37-b88b8c336e82 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:58.199504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bd6d1-bcfc-4376-9a37-b88b8c336e82 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:58.199765 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bd6d1-bcfc-4376-9a37-b88b8c336e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:58.202619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 479/960] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:58.208584 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa84c4-d5fc-455f-ac89-1bf716bde583 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:58.220528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:58.227113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-175bd6d1-bcfc-4376-9a37-b88b8c336e82 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:58.228823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-175bd6d1-bcfc-4376-9a37-b88b8c336e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:58.310660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 482/961] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:58.330653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa84c4-d5fc-455f-ac89-1bf716bde583 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:58.335654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bd6d1-bcfc-4376-9a37-b88b8c336e82 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:58.377979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704d46e-5558-4682-9f40-9b7f2d3244b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:58.378986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704d46e-5558-4682-9f40-9b7f2d3244b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:58.379457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704d46e-5558-4682-9f40-9b7f2d3244b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:58.379986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704d46e-5558-4682-9f40-9b7f2d3244b4 None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/6d62ec1463c14665b94e9b2d85e48e14/roles/c4fac6ed74ff415cb3e604007bcf0161` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:58.383416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704d46e-5558-4682-9f40-9b7f2d3244b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=6d62ec1463c14665b94e9b2d85e48e14, role_id=c4fac6ed74ff415cb3e604007bcf0161)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:58.385807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704d46e-5558-4682-9f40-9b7f2d3244b4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:58.437060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:58.442780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bd6d1-bcfc-4376-9a37-b88b8c336e82 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:58.462007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 480/962] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:05:58 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/6d62ec1463c14665b94e9b2d85e48e14/roles/c4fac6ed74ff415cb3e604007bcf0161 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:05:58.485672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398164b3-0332-4316-8180-7ec130895db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:58.486396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398164b3-0332-4316-8180-7ec130895db8 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:58.486689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398164b3-0332-4316-8180-7ec130895db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:58.487013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398164b3-0332-4316-8180-7ec130895db8 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:58.487724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398164b3-0332-4316-8180-7ec130895db8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:58.489375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398164b3-0332-4316-8180-7ec130895db8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:58.543279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.sql.core [None req-398164b3-0332-4316-8180-7ec130895db8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 01:05:58.543279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 06 01:05:58.543279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [parameters: {'user_id': '3d0a11a13df24be0aa0e2e6d424cc6bc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 06 01:05:58.543279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80524) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-398164b3-0332-4316-8180-7ec130895db8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result.read() Aug 06 01:05:58.550997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:05:58.551994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application yield resource Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application next(self.gen) Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise self.value Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application session.commit() Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 01:05:58.552940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.session.flush() Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application compat.raise_( Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 01:05:58.554663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 01:05:58.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application conn.query(q) Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application result.read() Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 01:05:58.559140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [parameters: {'user_id': '3d0a11a13df24be0aa0e2e6d424cc6bc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:58.561230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 06 01:05:58.563103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 06 01:05:58.570901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 06 01:05:58.570901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 06 01:05:58.570901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 01:05:58.570901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 01:05:58.570901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 06 01:05:58.570901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:05:58.570901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 483/963] 10.222.0.25 () {60 vars in 1131 bytes} [Sat Aug 6 01:05:58 2022] POST /identity/v3/users => generated 160 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:58.571326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704d46e-5558-4682-9f40-9b7f2d3244b4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:58.656590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:58.690562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704d46e-5558-4682-9f40-9b7f2d3244b4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:58.726870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2087d1-da8c-46fe-ba15-c5ce8b1080cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:58.727903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2087d1-da8c-46fe-ba15-c5ce8b1080cb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:58.728389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2087d1-da8c-46fe-ba15-c5ce8b1080cb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:58.728888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2087d1-da8c-46fe-ba15-c5ce8b1080cb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:58.737634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2087d1-da8c-46fe-ba15-c5ce8b1080cb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:58.741467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2087d1-da8c-46fe-ba15-c5ce8b1080cb None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-aa2087d1-da8c-46fe-ba15-c5ce8b1080cb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:05:58.751460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:58.752453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:58.753763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:58.753763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:58.753763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:05:58.753763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:05:58.753763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:05:58.753763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:05:58.753763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:05:58.753763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:05:58.753763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 01:05:58.753763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:58.756152 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 481/964] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:05:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:58.762505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398164b3-0332-4316-8180-7ec130895db8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:58.866438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398164b3-0332-4316-8180-7ec130895db8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:58.902204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5a22e-9a1b-4c99-8d01-d5ce2122f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:58.903111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5a22e-9a1b-4c99-8d01-d5ce2122f1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:58.903598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5a22e-9a1b-4c99-8d01-d5ce2122f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:58.904073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5a22e-9a1b-4c99-8d01-d5ce2122f1d4 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:58.905013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5a22e-9a1b-4c99-8d01-d5ce2122f1d4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:58.905538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5a22e-9a1b-4c99-8d01-d5ce2122f1d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:58.907415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5a22e-9a1b-4c99-8d01-d5ce2122f1d4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:58.946820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 484/965] 10.222.0.25 () {56 vars in 1101 bytes} [Sat Aug 6 01:05:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:59.525787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd60ed-d149-4ce0-8b37-86793bdab78e None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:59.526310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd60ed-d149-4ce0-8b37-86793bdab78e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:59.526310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd60ed-d149-4ce0-8b37-86793bdab78e None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:59.527809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 482/966] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:05:59.533008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef451e-bb46-4d3b-a654-8ea78c6175c1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:59.533422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef451e-bb46-4d3b-a654-8ea78c6175c1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:59.533589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef451e-bb46-4d3b-a654-8ea78c6175c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:59.552493 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-eeef451e-bb46-4d3b-a654-8ea78c6175c1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:05:59.554533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-eeef451e-bb46-4d3b-a654-8ea78c6175c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:05:59.565803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:59.629749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 485/967] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:05:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:05:59.651570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd60ed-d149-4ce0-8b37-86793bdab78e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:59.754373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd60ed-d149-4ce0-8b37-86793bdab78e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:59.780618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:05:59.789371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ca335-c210-4630-9fef-8d28827392c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:59.791905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ca335-c210-4630-9fef-8d28827392c0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:59.791905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ca335-c210-4630-9fef-8d28827392c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:59.791905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ca335-c210-4630-9fef-8d28827392c0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:59.791905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ca335-c210-4630-9fef-8d28827392c0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:59.792528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ca335-c210-4630-9fef-8d28827392c0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-e65ca335-c210-4630-9fef-8d28827392c0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:05:59.803084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:05:59.804937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:05:59.805802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:05:59.807851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 483/968] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:05:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:05:59.811217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef451e-bb46-4d3b-a654-8ea78c6175c1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:05:59.909656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef451e-bb46-4d3b-a654-8ea78c6175c1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:05:59.944591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6624d-d77a-4df3-8bf6-0bcb992099cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:05:59.945211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6624d-d77a-4df3-8bf6-0bcb992099cd None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:05:59.945489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6624d-d77a-4df3-8bf6-0bcb992099cd None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:05:59.945758 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6624d-d77a-4df3-8bf6-0bcb992099cd None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:05:59.946429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6624d-d77a-4df3-8bf6-0bcb992099cd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:05:59.946729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6624d-d77a-4df3-8bf6-0bcb992099cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:05:59.948114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6624d-d77a-4df3-8bf6-0bcb992099cd None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:05:59.953121 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 486/969] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Aug 6 01:05:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:05:59.959046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ca335-c210-4630-9fef-8d28827392c0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:00.059659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ca335-c210-4630-9fef-8d28827392c0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:00.093516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33db10-7caf-44f3-879d-ac3741773627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:00.094130 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33db10-7caf-44f3-879d-ac3741773627 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:00.094408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33db10-7caf-44f3-879d-ac3741773627 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:00.094680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33db10-7caf-44f3-879d-ac3741773627 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:00.095350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33db10-7caf-44f3-879d-ac3741773627 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:00.095653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33db10-7caf-44f3-879d-ac3741773627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:00.097035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33db10-7caf-44f3-879d-ac3741773627 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:00.102685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 484/970] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Aug 6 01:05:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:00.108483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6624d-d77a-4df3-8bf6-0bcb992099cd None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:00.114683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692e344-9ab0-4b2c-9881-09b6852ea289 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:00.114973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692e344-9ab0-4b2c-9881-09b6852ea289 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:00.115254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692e344-9ab0-4b2c-9881-09b6852ea289 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:00.116776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 485/971] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:06:00.121554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c074d80-ea35-455b-b12a-b6c7f1ab52a8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:00.121844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c074d80-ea35-455b-b12a-b6c7f1ab52a8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:00.121945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:00.122117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c074d80-ea35-455b-b12a-b6c7f1ab52a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:00.141150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6c074d80-ea35-455b-b12a-b6c7f1ab52a8 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:00.142338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6c074d80-ea35-455b-b12a-b6c7f1ab52a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:00.197136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:00.215161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 486/972] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:06:00.218318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6624d-d77a-4df3-8bf6-0bcb992099cd None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:00.237687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c074d80-ea35-455b-b12a-b6c7f1ab52a8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:00.255882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9806f0-3905-47b0-9ecf-e5a1e6b621c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:00.256506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9806f0-3905-47b0-9ecf-e5a1e6b621c1 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:00.256782 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9806f0-3905-47b0-9ecf-e5a1e6b621c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:00.257097 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9806f0-3905-47b0-9ecf-e5a1e6b621c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:00.257857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9806f0-3905-47b0-9ecf-e5a1e6b621c1 None admin] RBAC: Adding query filter params (interface=admin, service_id=b8f1e6f016304f5e8c29cdec46bc715a, region_id=RegionOne) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:00.258165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9806f0-3905-47b0-9ecf-e5a1e6b621c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:00.259525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9806f0-3905-47b0-9ecf-e5a1e6b621c1 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:00.264982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 487/973] 10.222.0.25 () {56 vars in 1249 bytes} [Sat Aug 6 01:06:00 2022] GET /identity/v3/endpoints?service_id=b8f1e6f016304f5e8c29cdec46bc715a&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:00.270304 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9806f0-3905-47b0-9ecf-e5a1e6b621c1 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:00.343019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c074d80-ea35-455b-b12a-b6c7f1ab52a8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:00.352087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:00.377347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9806f0-3905-47b0-9ecf-e5a1e6b621c1 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:00.383098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8a731-959b-4b43-a534-0840ebaddc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:00.383817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8a731-959b-4b43-a534-0840ebaddc9b None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:00.384165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8a731-959b-4b43-a534-0840ebaddc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:00.384510 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8a731-959b-4b43-a534-0840ebaddc9b None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:00.385280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8a731-959b-4b43-a534-0840ebaddc9b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:00.386779 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8a731-959b-4b43-a534-0840ebaddc9b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:00.411931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d00030-cee0-45b8-9103-2be481cb5917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:00.412587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d00030-cee0-45b8-9103-2be481cb5917 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:00.412880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d00030-cee0-45b8-9103-2be481cb5917 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:00.413184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d00030-cee0-45b8-9103-2be481cb5917 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:00.413838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d00030-cee0-45b8-9103-2be481cb5917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:00.415256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d00030-cee0-45b8-9103-2be481cb5917 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:00.421295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 488/974] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Aug 6 01:06:00 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:06:00.472469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 487/975] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:06:00 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:06:00.510077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ded8e4-d1b8-444d-b385-4dbb580355fe None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:00.510412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ded8e4-d1b8-444d-b385-4dbb580355fe None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:00.510642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ded8e4-d1b8-444d-b385-4dbb580355fe None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:00.512361 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 489/976] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:00.518041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f4a8d-01ea-46a2-b8f2-b137dd84f888 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:00.518361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f4a8d-01ea-46a2-b8f2-b137dd84f888 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:00.518649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f4a8d-01ea-46a2-b8f2-b137dd84f888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:00.542243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6e4f4a8d-01ea-46a2-b8f2-b137dd84f888 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:00.544720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6e4f4a8d-01ea-46a2-b8f2-b137dd84f888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:00.598506 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:00.637356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 488/977] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:00.660636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ded8e4-d1b8-444d-b385-4dbb580355fe None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:00.766095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ded8e4-d1b8-444d-b385-4dbb580355fe None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:00.802886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936261e-9caf-4f4e-b715-23e82dc9d355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:00.803534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936261e-9caf-4f4e-b715-23e82dc9d355 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:00.804399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936261e-9caf-4f4e-b715-23e82dc9d355 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:00.804808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936261e-9caf-4f4e-b715-23e82dc9d355 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:00.812655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:00.816852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936261e-9caf-4f4e-b715-23e82dc9d355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4936261e-9caf-4f4e-b715-23e82dc9d355 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:00.822424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:00.829779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:00.830644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 490/978] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Aug 6 01:06:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:00.831372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f4a8d-01ea-46a2-b8f2-b137dd84f888 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:00.931713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f4a8d-01ea-46a2-b8f2-b137dd84f888 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:00.966128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110aee5-37f6-41f7-bcb3-e84dcf73906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:00.966749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110aee5-37f6-41f7-bcb3-e84dcf73906a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:00.967032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110aee5-37f6-41f7-bcb3-e84dcf73906a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:00.967308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110aee5-37f6-41f7-bcb3-e84dcf73906a None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:00.968067 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110aee5-37f6-41f7-bcb3-e84dcf73906a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:00.968375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110aee5-37f6-41f7-bcb3-e84dcf73906a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:00.969753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110aee5-37f6-41f7-bcb3-e84dcf73906a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:00.982532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 489/979] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Aug 6 01:06:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:00.983489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936261e-9caf-4f4e-b715-23e82dc9d355 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:01.085766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936261e-9caf-4f4e-b715-23e82dc9d355 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:01.120238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951b49c-04d7-4cbd-9562-dc22dd899ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:01.120882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951b49c-04d7-4cbd-9562-dc22dd899ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:01.121215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951b49c-04d7-4cbd-9562-dc22dd899ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:01.121496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951b49c-04d7-4cbd-9562-dc22dd899ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:01.131497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951b49c-04d7-4cbd-9562-dc22dd899ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:01.133028 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951b49c-04d7-4cbd-9562-dc22dd899ce5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-8951b49c-04d7-4cbd-9562-dc22dd899ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:06:01.148340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:01.149258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:01.156325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:06:01.156325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:06:01.156325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:01.156325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:01.156325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 491/980] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:01.156607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110aee5-37f6-41f7-bcb3-e84dcf73906a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:01.192677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:01.260365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110aee5-37f6-41f7-bcb3-e84dcf73906a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:01.294425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab6999-e73c-462e-9f46-ad536bbf48c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:01.295036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab6999-e73c-462e-9f46-ad536bbf48c9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:01.295364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab6999-e73c-462e-9f46-ad536bbf48c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:01.295642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab6999-e73c-462e-9f46-ad536bbf48c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:01.296358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab6999-e73c-462e-9f46-ad536bbf48c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:01.296663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab6999-e73c-462e-9f46-ad536bbf48c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:01.298076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab6999-e73c-462e-9f46-ad536bbf48c9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:01.313707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 490/981] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:01.315049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951b49c-04d7-4cbd-9562-dc22dd899ce5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:01.364863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:01.415125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951b49c-04d7-4cbd-9562-dc22dd899ce5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:01.448604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc1018-b066-4a11-9b23-5d4dde044538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:01.449378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc1018-b066-4a11-9b23-5d4dde044538 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:01.449753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc1018-b066-4a11-9b23-5d4dde044538 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:01.450124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc1018-b066-4a11-9b23-5d4dde044538 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:01.457691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc1018-b066-4a11-9b23-5d4dde044538 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:01.459277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc1018-b066-4a11-9b23-5d4dde044538 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-88cc1018-b066-4a11-9b23-5d4dde044538 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:06:01.470367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:01.471467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:01.472647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:01.472647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:01.472647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:06:01.472647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:06:01.472647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:06:01.472647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:06:01.472647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:06:01.472647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:06:01.472647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 01:06:01.472647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:01.478774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 492/982] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Aug 6 01:06:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:01.479794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab6999-e73c-462e-9f46-ad536bbf48c9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:01.575096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:01.591478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab6999-e73c-462e-9f46-ad536bbf48c9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:01.627377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbd5b9-7b68-4705-ad00-43575c162305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:01.628039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbd5b9-7b68-4705-ad00-43575c162305 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:01.628324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbd5b9-7b68-4705-ad00-43575c162305 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:01.628609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbd5b9-7b68-4705-ad00-43575c162305 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:01.629440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbd5b9-7b68-4705-ad00-43575c162305 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:01.629756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbd5b9-7b68-4705-ad00-43575c162305 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:01.631188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbd5b9-7b68-4705-ad00-43575c162305 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:01.651103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 491/983] 10.222.0.25 () {56 vars in 1149 bytes} [Sat Aug 6 01:06:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:01.659806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc1018-b066-4a11-9b23-5d4dde044538 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:01.755186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc1018-b066-4a11-9b23-5d4dde044538 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:01.765571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:01.788058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3f1b0-cac8-4f35-8993-114379e1325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:01.789214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3f1b0-cac8-4f35-8993-114379e1325a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:01.789491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3f1b0-cac8-4f35-8993-114379e1325a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:01.789763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3f1b0-cac8-4f35-8993-114379e1325a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:01.860738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3f1b0-cac8-4f35-8993-114379e1325a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:01.862198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3f1b0-cac8-4f35-8993-114379e1325a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:01.958585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 493/984] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:01 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:06:01.964220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbd5b9-7b68-4705-ad00-43575c162305 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:02.065931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbd5b9-7b68-4705-ad00-43575c162305 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:02.099698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac6b50-5c37-4223-b915-24d13c0b2ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:02.100386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac6b50-5c37-4223-b915-24d13c0b2ece None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.101181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac6b50-5c37-4223-b915-24d13c0b2ece None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.101462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac6b50-5c37-4223-b915-24d13c0b2ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.111614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac6b50-5c37-4223-b915-24d13c0b2ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:02.113083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac6b50-5c37-4223-b915-24d13c0b2ece None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-43ac6b50-5c37-4223-b915-24d13c0b2ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:06:02.128200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:02.129121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:02.130147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:06:02.130147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:06:02.130147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:02.130147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:02.130147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 492/985] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:02.136688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3f1b0-cac8-4f35-8993-114379e1325a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:02.148854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:02.236234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3f1b0-cac8-4f35-8993-114379e1325a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:02.269358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1eaa7a-e187-4360-b17f-41e1c66f675f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:02.269980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eaa7a-e187-4360-b17f-41e1c66f675f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.270262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eaa7a-e187-4360-b17f-41e1c66f675f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.270534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1eaa7a-e187-4360-b17f-41e1c66f675f None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.271221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eaa7a-e187-4360-b17f-41e1c66f675f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:02.271524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eaa7a-e187-4360-b17f-41e1c66f675f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:02.273426 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1eaa7a-e187-4360-b17f-41e1c66f675f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:02.284662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 494/986] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:02.290604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac6b50-5c37-4223-b915-24d13c0b2ece None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:02.295382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:02.392685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac6b50-5c37-4223-b915-24d13c0b2ece None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:02.426891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cbd37-afc5-4b4e-a1f0-94e50a0f3df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:02.427531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cbd37-afc5-4b4e-a1f0-94e50a0f3df2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.427842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cbd37-afc5-4b4e-a1f0-94e50a0f3df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.428128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cbd37-afc5-4b4e-a1f0-94e50a0f3df2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.434794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cbd37-afc5-4b4e-a1f0-94e50a0f3df2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:02.436265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cbd37-afc5-4b4e-a1f0-94e50a0f3df2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-eb6cbd37-afc5-4b4e-a1f0-94e50a0f3df2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:06:02.447566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:02.448520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:02.449454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:06:02.449454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:06:02.449454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:06:02.449454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:06:02.449454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:06:02.449454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:06:02.449454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:06:02.449454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:02.449454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 493/987] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:02.455650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd7299-7324-4dc5-877b-b9f96272f5bf None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.455911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cbd37-afc5-4b4e-a1f0-94e50a0f3df2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:02.456016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd7299-7324-4dc5-877b-b9f96272f5bf None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.456235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd7299-7324-4dc5-877b-b9f96272f5bf None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.457578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 495/988] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:02.462981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f481718-be22-404b-ae37-8b66f2d66f77 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.463270 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f481718-be22-404b-ae37-8b66f2d66f77 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.463547 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f481718-be22-404b-ae37-8b66f2d66f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.483057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2f481718-be22-404b-ae37-8b66f2d66f77 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:02.484260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2f481718-be22-404b-ae37-8b66f2d66f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:02.528859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:02.556587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:02.562299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 496/989] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:06:02.563535 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb69e1b-4443-4b52-855a-4ec86b1a7bf8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.563849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb69e1b-4443-4b52-855a-4ec86b1a7bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.564164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb69e1b-4443-4b52-855a-4ec86b1a7bf8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.570540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 497/990] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:02.570833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1b192-72e4-49c9-85cc-bc3438671de6 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.571078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1b192-72e4-49c9-85cc-bc3438671de6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.571342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1b192-72e4-49c9-85cc-bc3438671de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.571452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cbd37-afc5-4b4e-a1f0-94e50a0f3df2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:02.589956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d7b1b192-72e4-49c9-85cc-bc3438671de6 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:02.592443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d7b1b192-72e4-49c9-85cc-bc3438671de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:02.609285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56689444-986d-4b14-a605-83026f4cf09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:02.609909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56689444-986d-4b14-a605-83026f4cf09d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.610186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56689444-986d-4b14-a605-83026f4cf09d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.610458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56689444-986d-4b14-a605-83026f4cf09d None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.611200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56689444-986d-4b14-a605-83026f4cf09d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:02.611508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56689444-986d-4b14-a605-83026f4cf09d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:02.613080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56689444-986d-4b14-a605-83026f4cf09d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:02.625346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 494/991] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:06:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:02.626144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56689444-986d-4b14-a605-83026f4cf09d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:02.658376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 498/992] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:06:02.659439 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1b192-72e4-49c9-85cc-bc3438671de6 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:02.727207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56689444-986d-4b14-a605-83026f4cf09d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:02.749788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:02.755016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1b192-72e4-49c9-85cc-bc3438671de6 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:02.760569 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfc186-157b-4544-9467-67f26126fb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:02.764960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfc186-157b-4544-9467-67f26126fb98 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.764960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfc186-157b-4544-9467-67f26126fb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.764960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfc186-157b-4544-9467-67f26126fb98 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.764960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfc186-157b-4544-9467-67f26126fb98 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:02.764960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfc186-157b-4544-9467-67f26126fb98 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-a2cfc186-157b-4544-9467-67f26126fb98 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:02.774048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:02.775765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:02.776790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 495/993] 10.222.0.25 () {56 vars in 1126 bytes} [Sat Aug 6 01:06:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:02.777655 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfc186-157b-4544-9467-67f26126fb98 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:02.787839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:02.795008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686141b8-0404-4eed-bec4-5611d3628b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:02.795632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686141b8-0404-4eed-bec4-5611d3628b20 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.795948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686141b8-0404-4eed-bec4-5611d3628b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.796226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686141b8-0404-4eed-bec4-5611d3628b20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.797017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686141b8-0404-4eed-bec4-5611d3628b20 None admin] RBAC: Adding query filter params (role.id=c4fac6ed74ff415cb3e604007bcf0161, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=6d62ec1463c14665b94e9b2d85e48e14) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:02.797325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686141b8-0404-4eed-bec4-5611d3628b20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:02.798762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686141b8-0404-4eed-bec4-5611d3628b20 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:02.804662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 499/994] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:06:02 2022] GET /identity/v3/role_assignments?user.id=6d62ec1463c14665b94e9b2d85e48e14&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=c4fac6ed74ff415cb3e604007bcf0161 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:02.805542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686141b8-0404-4eed-bec4-5611d3628b20 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:02.901884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfc186-157b-4544-9467-67f26126fb98 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:02.927227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686141b8-0404-4eed-bec4-5611d3628b20 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:02.941829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c84ace-3ba7-4106-ba33-c7c83bc4b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:02.942602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c84ace-3ba7-4106-ba33-c7c83bc4b83f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.942939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c84ace-3ba7-4106-ba33-c7c83bc4b83f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.943282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c84ace-3ba7-4106-ba33-c7c83bc4b83f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.943955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c84ace-3ba7-4106-ba33-c7c83bc4b83f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:02.946016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c84ace-3ba7-4106-ba33-c7c83bc4b83f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-24c84ace-3ba7-4106-ba33-c7c83bc4b83f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:02.956139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:02.957333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:02.958300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 496/995] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:06:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:02.964306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c84ace-3ba7-4106-ba33-c7c83bc4b83f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:02.974326 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c28a7-8877-4712-8d53-2b38ec4c84bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:02.974961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c28a7-8877-4712-8d53-2b38ec4c84bf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:02.975252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c28a7-8877-4712-8d53-2b38ec4c84bf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:02.975528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c28a7-8877-4712-8d53-2b38ec4c84bf None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:02.976322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c28a7-8877-4712-8d53-2b38ec4c84bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:02.976629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c28a7-8877-4712-8d53-2b38ec4c84bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:02.978002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c28a7-8877-4712-8d53-2b38ec4c84bf None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:02.983398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 500/996] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Aug 6 01:06:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:02.989286 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c28a7-8877-4712-8d53-2b38ec4c84bf None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:03.032870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:03.055693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:03.081629 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c84ace-3ba7-4106-ba33-c7c83bc4b83f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:03.101381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c28a7-8877-4712-8d53-2b38ec4c84bf None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:03.117273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2140b1-3632-4bcc-b23b-3b6c7095b0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:03.117898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2140b1-3632-4bcc-b23b-3b6c7095b0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:03.118183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2140b1-3632-4bcc-b23b-3b6c7095b0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:03.118463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2140b1-3632-4bcc-b23b-3b6c7095b0d4 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:03.119147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2140b1-3632-4bcc-b23b-3b6c7095b0d4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:03.119460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2140b1-3632-4bcc-b23b-3b6c7095b0d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:03.120909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2140b1-3632-4bcc-b23b-3b6c7095b0d4 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:03.130172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 497/997] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:06:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:03.131149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2140b1-3632-4bcc-b23b-3b6c7095b0d4 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:03.148378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88486842-1a29-4da3-9a56-e06ee9f5faa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:03.149024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88486842-1a29-4da3-9a56-e06ee9f5faa2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:03.149310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88486842-1a29-4da3-9a56-e06ee9f5faa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:03.149590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88486842-1a29-4da3-9a56-e06ee9f5faa2 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:03.150275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88486842-1a29-4da3-9a56-e06ee9f5faa2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:03.150587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88486842-1a29-4da3-9a56-e06ee9f5faa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:03.152026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88486842-1a29-4da3-9a56-e06ee9f5faa2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:03.161919 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 501/998] 10.222.0.25 () {56 vars in 1122 bytes} [Sat Aug 6 01:06:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:03.163012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88486842-1a29-4da3-9a56-e06ee9f5faa2 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:03.245380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2140b1-3632-4bcc-b23b-3b6c7095b0d4 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:03.262101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:03.273198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88486842-1a29-4da3-9a56-e06ee9f5faa2 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:03.279516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9879d92-b9cb-4998-9f37-005f079cd3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:03.280171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9879d92-b9cb-4998-9f37-005f079cd3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:03.280455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9879d92-b9cb-4998-9f37-005f079cd3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:03.280735 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9879d92-b9cb-4998-9f37-005f079cd3f1 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:03.281428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9879d92-b9cb-4998-9f37-005f079cd3f1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:03.281737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9879d92-b9cb-4998-9f37-005f079cd3f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:03.283113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9879d92-b9cb-4998-9f37-005f079cd3f1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:03.293482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 498/999] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:06:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:03.294822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9879d92-b9cb-4998-9f37-005f079cd3f1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:03.302326 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:03.318848 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e41da-b3c2-4324-bd5d-bb296a1d13b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:03.319520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e41da-b3c2-4324-bd5d-bb296a1d13b3 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:03.319827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e41da-b3c2-4324-bd5d-bb296a1d13b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:03.320114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e41da-b3c2-4324-bd5d-bb296a1d13b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:03.320831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e41da-b3c2-4324-bd5d-bb296a1d13b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:03.322213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e41da-b3c2-4324-bd5d-bb296a1d13b3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:03.395474 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9879d92-b9cb-4998-9f37-005f079cd3f1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:03.428696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab21c9-8c96-4284-9c64-aa305676200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:03.429998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab21c9-8c96-4284-9c64-aa305676200d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:03.430282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab21c9-8c96-4284-9c64-aa305676200d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:03.430558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab21c9-8c96-4284-9c64-aa305676200d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:03.431305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab21c9-8c96-4284-9c64-aa305676200d None admin] RBAC: Adding query filter params (interface=public, service_id=3335dbc6ceaa4047ba11959fc35486dc, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:03.439980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab21c9-8c96-4284-9c64-aa305676200d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:03.439980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab21c9-8c96-4284-9c64-aa305676200d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:03.439980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 499/1000] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Aug 6 01:06:03 2022] GET /identity/v3/endpoints?service_id=3335dbc6ceaa4047ba11959fc35486dc&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:03.449977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 502/1001] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:06:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:06:03.450062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab21c9-8c96-4284-9c64-aa305676200d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:03.497182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:03.560468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab21c9-8c96-4284-9c64-aa305676200d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:03.605225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abffe72-2270-4a55-9742-9d149b594968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:03.606113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abffe72-2270-4a55-9742-9d149b594968 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:03.606562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abffe72-2270-4a55-9742-9d149b594968 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:03.606838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abffe72-2270-4a55-9742-9d149b594968 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:03.607493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abffe72-2270-4a55-9742-9d149b594968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:03.608928 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abffe72-2270-4a55-9742-9d149b594968 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:03.615387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 500/1002] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Aug 6 01:06:03 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:06:04.852402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcd84d-7004-45d2-be21-7e365902602e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:04.852909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcd84d-7004-45d2-be21-7e365902602e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:04.852909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcd84d-7004-45d2-be21-7e365902602e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:04.854358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 503/1003] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:04.859365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28afd2-a775-4b04-8da3-4c71e0d9358c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:04.859757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28afd2-a775-4b04-8da3-4c71e0d9358c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:04.859963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28afd2-a775-4b04-8da3-4c71e0d9358c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:04.878359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5d28afd2-a775-4b04-8da3-4c71e0d9358c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:04.879555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5d28afd2-a775-4b04-8da3-4c71e0d9358c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:04.952806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 501/1004] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:04.974924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcd84d-7004-45d2-be21-7e365902602e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:05.076093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcd84d-7004-45d2-be21-7e365902602e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:05.109236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514920f-3c63-4788-9191-8381a511389c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:05.109801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514920f-3c63-4788-9191-8381a511389c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:05.110079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514920f-3c63-4788-9191-8381a511389c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:05.110347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514920f-3c63-4788-9191-8381a511389c None admin] PATH_INFO: `/v3/services/image` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:05.110963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514920f-3c63-4788-9191-8381a511389c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:05.112382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514920f-3c63-4788-9191-8381a511389c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4514920f-3c63-4788-9191-8381a511389c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:05.121302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:05.122298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:06:05.123132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:05.130648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 504/1005] 10.222.0.25 () {56 vars in 1105 bytes} [Sat Aug 6 01:06:04 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:05.130795 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28afd2-a775-4b04-8da3-4c71e0d9358c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:05.163272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:05.230003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28afd2-a775-4b04-8da3-4c71e0d9358c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:05.264140 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4427806-10f5-4b80-b781-7ed961998a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:05.264763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4427806-10f5-4b80-b781-7ed961998a50 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:05.265043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4427806-10f5-4b80-b781-7ed961998a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:05.265322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4427806-10f5-4b80-b781-7ed961998a50 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:05.265994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4427806-10f5-4b80-b781-7ed961998a50 None admin] RBAC: Adding query filter params (name=image) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:05.266298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4427806-10f5-4b80-b781-7ed961998a50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:05.267646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4427806-10f5-4b80-b781-7ed961998a50 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:05.272507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 502/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:05.277997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514920f-3c63-4788-9191-8381a511389c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:05.326921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:05.379479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514920f-3c63-4788-9191-8381a511389c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:05.413788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cfa38-d3ce-4bea-ba2d-a9cd3271669a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:05.414403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cfa38-d3ce-4bea-ba2d-a9cd3271669a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:05.414683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cfa38-d3ce-4bea-ba2d-a9cd3271669a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:05.414954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cfa38-d3ce-4bea-ba2d-a9cd3271669a None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:05.415625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cfa38-d3ce-4bea-ba2d-a9cd3271669a None admin] RBAC: Adding query filter params (type=image) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:05.415956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cfa38-d3ce-4bea-ba2d-a9cd3271669a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:05.417369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cfa38-d3ce-4bea-ba2d-a9cd3271669a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:05.421962 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 505/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:05 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:05.465813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016b2fd-03ff-4239-bfa9-ed1baef49291 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:05.466124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016b2fd-03ff-4239-bfa9-ed1baef49291 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:05.466439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016b2fd-03ff-4239-bfa9-ed1baef49291 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:05.467970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 503/1008] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:05.473864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1f64c-a642-435c-a80c-79b6216d9f39 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:05.474192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1f64c-a642-435c-a80c-79b6216d9f39 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:05.474442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1f64c-a642-435c-a80c-79b6216d9f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:05.493842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-64c1f64c-a642-435c-a80c-79b6216d9f39 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:05.495082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-64c1f64c-a642-435c-a80c-79b6216d9f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:05.574793 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 506/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:05.597733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016b2fd-03ff-4239-bfa9-ed1baef49291 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:05.652565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec1842-f633-4125-81a1-35d4beb22f53 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:05.652855 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec1842-f633-4125-81a1-35d4beb22f53 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:05.653044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec1842-f633-4125-81a1-35d4beb22f53 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:05.654559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 507/1010] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:06:05.659955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585268c8-0d26-44c6-99f6-1f72d4aaa98c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:05.660650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585268c8-0d26-44c6-99f6-1f72d4aaa98c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:05.660943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585268c8-0d26-44c6-99f6-1f72d4aaa98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:05.685742 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-585268c8-0d26-44c6-99f6-1f72d4aaa98c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:05.687231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-585268c8-0d26-44c6-99f6-1f72d4aaa98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:05.701204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:05.706645 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016b2fd-03ff-4239-bfa9-ed1baef49291 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:05.724943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:05.741102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66557ead-6b55-402a-8ea8-202074f15d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:05.742410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557ead-6b55-402a-8ea8-202074f15d77 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:05.742690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557ead-6b55-402a-8ea8-202074f15d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:05.742964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557ead-6b55-402a-8ea8-202074f15d77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:05.744150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66557ead-6b55-402a-8ea8-202074f15d77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:05.745662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66557ead-6b55-402a-8ea8-202074f15d77 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:05.747814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 504/1011] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:05.752850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66557ead-6b55-402a-8ea8-202074f15d77 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:05.764068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 508/1012] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:05.786127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585268c8-0d26-44c6-99f6-1f72d4aaa98c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:05.865851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66557ead-6b55-402a-8ea8-202074f15d77 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:05.890991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585268c8-0d26-44c6-99f6-1f72d4aaa98c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:05.904877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6a442-c88c-4bf8-a1de-e772cea045c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:05.905425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6a442-c88c-4bf8-a1de-e772cea045c2 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:05.905706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6a442-c88c-4bf8-a1de-e772cea045c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:05.905986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6a442-c88c-4bf8-a1de-e772cea045c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:05.906709 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6a442-c88c-4bf8-a1de-e772cea045c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:05.908155 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda6a442-c88c-4bf8-a1de-e772cea045c2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:05.938966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89fc3f-3b27-45c6-820d-13ea7e126ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:05.939711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89fc3f-3b27-45c6-820d-13ea7e126ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:05.940094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89fc3f-3b27-45c6-820d-13ea7e126ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:05.940372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89fc3f-3b27-45c6-820d-13ea7e126ce8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:05.941018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89fc3f-3b27-45c6-820d-13ea7e126ce8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:05.942908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89fc3f-3b27-45c6-820d-13ea7e126ce8 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-cf89fc3f-3b27-45c6-820d-13ea7e126ce8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:05.952871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:05.954310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:05.955262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 509/1013] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:06:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:05.960697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89fc3f-3b27-45c6-820d-13ea7e126ce8 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:05.995235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 505/1014] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:06:05 2022] POST /identity/v3/projects => generated 311 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:06:05.995946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:06.075258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89fc3f-3b27-45c6-820d-13ea7e126ce8 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:06.113086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61fe3a-6227-4720-bd1e-89a23f094b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:06.113726 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61fe3a-6227-4720-bd1e-89a23f094b97 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:06.114006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61fe3a-6227-4720-bd1e-89a23f094b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:06.114276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61fe3a-6227-4720-bd1e-89a23f094b97 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:06.114977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61fe3a-6227-4720-bd1e-89a23f094b97 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:06.115276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61fe3a-6227-4720-bd1e-89a23f094b97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:06.116731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61fe3a-6227-4720-bd1e-89a23f094b97 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:06.126869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 510/1015] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:06:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:06.128595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6a442-c88c-4bf8-a1de-e772cea045c2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:06.149400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:06.235220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6a442-c88c-4bf8-a1de-e772cea045c2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:06.269409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6dada-7293-4800-ae7e-8bb071104175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:06.270330 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6dada-7293-4800-ae7e-8bb071104175 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:06.270798 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6dada-7293-4800-ae7e-8bb071104175 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:06.271267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6dada-7293-4800-ae7e-8bb071104175 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:06.272167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6dada-7293-4800-ae7e-8bb071104175 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:06.273195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6dada-7293-4800-ae7e-8bb071104175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:06.274752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6dada-7293-4800-ae7e-8bb071104175 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:06.280091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 506/1016] 10.222.0.25 () {56 vars in 1112 bytes} [Sat Aug 6 01:06:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:06.285616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61fe3a-6227-4720-bd1e-89a23f094b97 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:06.381208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:06.386103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61fe3a-6227-4720-bd1e-89a23f094b97 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:06.426818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564912cd-8111-4d4d-8fec-443890817c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:06.427443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564912cd-8111-4d4d-8fec-443890817c94 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:06.427723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564912cd-8111-4d4d-8fec-443890817c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:06.428021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564912cd-8111-4d4d-8fec-443890817c94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:06.428751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564912cd-8111-4d4d-8fec-443890817c94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:06.430093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564912cd-8111-4d4d-8fec-443890817c94 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:06.495137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 511/1017] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:06:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:06:07.423437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5de34-de04-447e-912c-fb3ab5ece3c6 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:07.423976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5de34-de04-447e-912c-fb3ab5ece3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:07.424056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5de34-de04-447e-912c-fb3ab5ece3c6 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:07.425498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 507/1018] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:07.430839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06852899-870f-4c71-8d17-ba3accc9b92d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:07.431224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06852899-870f-4c71-8d17-ba3accc9b92d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:07.431385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06852899-870f-4c71-8d17-ba3accc9b92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:07.449567 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-06852899-870f-4c71-8d17-ba3accc9b92d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:07.450727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-06852899-870f-4c71-8d17-ba3accc9b92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:07.523823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 512/1019] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:07 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:07.545779 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5de34-de04-447e-912c-fb3ab5ece3c6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:07.638466 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:07.657262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5de34-de04-447e-912c-fb3ab5ece3c6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:07.691897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bec6b5-5eed-4f27-88b6-c77b08b6b1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:07.692461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bec6b5-5eed-4f27-88b6-c77b08b6b1a6 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:07.692744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bec6b5-5eed-4f27-88b6-c77b08b6b1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:07.693024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bec6b5-5eed-4f27-88b6-c77b08b6b1a6 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:07.693757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bec6b5-5eed-4f27-88b6-c77b08b6b1a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:07.695798 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bec6b5-5eed-4f27-88b6-c77b08b6b1a6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:07.795358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 508/1020] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:06:07 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:06:08.050319 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a242d32-f296-4dc5-ae19-6b69c965b1ca None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:08.050601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a242d32-f296-4dc5-ae19-6b69c965b1ca None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:08.050877 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a242d32-f296-4dc5-ae19-6b69c965b1ca None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:08.052349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 513/1021] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:08.058062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d396bbc-4bae-46b3-9de0-388b9e53d94a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:08.058349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d396bbc-4bae-46b3-9de0-388b9e53d94a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:08.058628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d396bbc-4bae-46b3-9de0-388b9e53d94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:08.076726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2d396bbc-4bae-46b3-9de0-388b9e53d94a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:08.077899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2d396bbc-4bae-46b3-9de0-388b9e53d94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:08.151166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 509/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:08.176291 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a242d32-f296-4dc5-ae19-6b69c965b1ca None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:08.258474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:08.278837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a242d32-f296-4dc5-ae19-6b69c965b1ca None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:08.313407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc15eb-9fa7-441c-b47d-52c406c0bb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:08.314364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc15eb-9fa7-441c-b47d-52c406c0bb9c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:08.314821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc15eb-9fa7-441c-b47d-52c406c0bb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:08.315280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc15eb-9fa7-441c-b47d-52c406c0bb9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:08.316668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc15eb-9fa7-441c-b47d-52c406c0bb9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:08.318397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc15eb-9fa7-441c-b47d-52c406c0bb9c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:08.320513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 514/1023] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:08.326160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d396bbc-4bae-46b3-9de0-388b9e53d94a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:08.392732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:08.430722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d396bbc-4bae-46b3-9de0-388b9e53d94a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:08.465762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d82f7-3640-48e0-945b-ddc78532652f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:08.466382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d82f7-3640-48e0-945b-ddc78532652f None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:08.466663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d82f7-3640-48e0-945b-ddc78532652f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:08.466937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d82f7-3640-48e0-945b-ddc78532652f None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:08.467634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d82f7-3640-48e0-945b-ddc78532652f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:08.469092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d82f7-3640-48e0-945b-ddc78532652f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:08.560563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 510/1024] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:06:08 2022] POST /identity/v3/users => generated 285 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:06:08.606061 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405afb0-782c-41bb-b224-9c93484866d0 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:08.606587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405afb0-782c-41bb-b224-9c93484866d0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:08.606587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405afb0-782c-41bb-b224-9c93484866d0 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:08.608177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 515/1025] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:08.614074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c52ef-0313-422a-bc02-b519f2900fe9 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:08.614429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c52ef-0313-422a-bc02-b519f2900fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:08.614699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c52ef-0313-422a-bc02-b519f2900fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:08.634269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-146c52ef-0313-422a-bc02-b519f2900fe9 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:08.635560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-146c52ef-0313-422a-bc02-b519f2900fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:08.709718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 511/1026] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:08.732967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405afb0-782c-41bb-b224-9c93484866d0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:08.836479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405afb0-782c-41bb-b224-9c93484866d0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:08.869177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41124bce-f0f8-4ec2-a006-cfcb080de2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:08.878128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41124bce-f0f8-4ec2-a006-cfcb080de2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:08.878128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41124bce-f0f8-4ec2-a006-cfcb080de2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:08.878128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41124bce-f0f8-4ec2-a006-cfcb080de2f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:08.880577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41124bce-f0f8-4ec2-a006-cfcb080de2f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-41124bce-f0f8-4ec2-a006-cfcb080de2f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:08.886017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:08.887022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:08.888393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 516/1027] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Aug 6 01:06:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:08.893452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c52ef-0313-422a-bc02-b519f2900fe9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:08.954453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:08.999121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c52ef-0313-422a-bc02-b519f2900fe9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:09.032636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ba4fd-0fff-4e1c-85a6-351e65710e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:09.033256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ba4fd-0fff-4e1c-85a6-351e65710e57 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:09.033536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ba4fd-0fff-4e1c-85a6-351e65710e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:09.033806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ba4fd-0fff-4e1c-85a6-351e65710e57 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:09.034513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ba4fd-0fff-4e1c-85a6-351e65710e57 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:09.034812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ba4fd-0fff-4e1c-85a6-351e65710e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:09.036190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ba4fd-0fff-4e1c-85a6-351e65710e57 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:09.048607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 512/1028] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Aug 6 01:06:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:09.049577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41124bce-f0f8-4ec2-a006-cfcb080de2f6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:09.061529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:09.149728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41124bce-f0f8-4ec2-a006-cfcb080de2f6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:09.182585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0080a-bf4e-48c9-a3ca-80e6c4aecda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:09.183224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0080a-bf4e-48c9-a3ca-80e6c4aecda0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:09.183514 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0080a-bf4e-48c9-a3ca-80e6c4aecda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:09.183770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0080a-bf4e-48c9-a3ca-80e6c4aecda0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:09.193512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0080a-bf4e-48c9-a3ca-80e6c4aecda0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:09.195572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0080a-bf4e-48c9-a3ca-80e6c4aecda0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-fac0080a-bf4e-48c9-a3ca-80e6c4aecda0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:06:09.210651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:09.211582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:09.213036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:06:09.213036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:06:09.213036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:09.213036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:09.213036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 517/1029] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:09.217782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ba4fd-0fff-4e1c-85a6-351e65710e57 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:09.318221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ba4fd-0fff-4e1c-85a6-351e65710e57 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:09.334395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:09.352525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b017f-0e0d-400e-8c03-8e2689406f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:09.353265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b017f-0e0d-400e-8c03-8e2689406f9d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:09.353610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b017f-0e0d-400e-8c03-8e2689406f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:09.353947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b017f-0e0d-400e-8c03-8e2689406f9d None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:09.354701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b017f-0e0d-400e-8c03-8e2689406f9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:09.355080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b017f-0e0d-400e-8c03-8e2689406f9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:09.356528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b017f-0e0d-400e-8c03-8e2689406f9d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:09.369473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 513/1030] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:09.374966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0080a-bf4e-48c9-a3ca-80e6c4aecda0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:09.434052 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:09.473587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0080a-bf4e-48c9-a3ca-80e6c4aecda0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:09.507201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2e565-a124-4715-9c76-db50f6807aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:09.507874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2e565-a124-4715-9c76-db50f6807aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:09.508158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2e565-a124-4715-9c76-db50f6807aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:09.508428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2e565-a124-4715-9c76-db50f6807aa6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:09.515836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2e565-a124-4715-9c76-db50f6807aa6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:09.517269 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2e565-a124-4715-9c76-db50f6807aa6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-77b2e565-a124-4715-9c76-db50f6807aa6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:06:09.528109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:09.529117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:09.530113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:09.530113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:09.530113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:06:09.530113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:06:09.530113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:06:09.530113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:06:09.530113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:06:09.530113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:06:09.530113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:06:09.530113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:09.532104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 518/1031] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Aug 6 01:06:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:09.535000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b017f-0e0d-400e-8c03-8e2689406f9d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:09.648063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b017f-0e0d-400e-8c03-8e2689406f9d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:09.681983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce823d-dc75-4823-9dbc-cd28cff92c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:09.682596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce823d-dc75-4823-9dbc-cd28cff92c08 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:09.682876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce823d-dc75-4823-9dbc-cd28cff92c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:09.683147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce823d-dc75-4823-9dbc-cd28cff92c08 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:09.683967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce823d-dc75-4823-9dbc-cd28cff92c08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:09.684342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce823d-dc75-4823-9dbc-cd28cff92c08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:09.685685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce823d-dc75-4823-9dbc-cd28cff92c08 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:09.702669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 514/1032] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Aug 6 01:06:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:09.708535 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2e565-a124-4715-9c76-db50f6807aa6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:09.812990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2e565-a124-4715-9c76-db50f6807aa6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:09.815160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e05174-006b-4597-bc23-60b4217eb0de None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:09.815446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e05174-006b-4597-bc23-60b4217eb0de None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:09.815724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e05174-006b-4597-bc23-60b4217eb0de None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:09.818247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 515/1033] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:06:09.823256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ed468-b66f-42cf-af08-2398a14af75c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:09.823543 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ed468-b66f-42cf-af08-2398a14af75c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:09.823840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ed468-b66f-42cf-af08-2398a14af75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:09.837015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:09.842331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-253ed468-b66f-42cf-af08-2398a14af75c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:09.843585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-253ed468-b66f-42cf-af08-2398a14af75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:09.852505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0568b4-8947-4489-81bc-d27e5e5da60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:09.853506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0568b4-8947-4489-81bc-d27e5e5da60e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:09.853506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0568b4-8947-4489-81bc-d27e5e5da60e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:09.854089 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0568b4-8947-4489-81bc-d27e5e5da60e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:09.873491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:09.919328 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 516/1034] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:06:09.927195 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0568b4-8947-4489-81bc-d27e5e5da60e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:09.928669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0568b4-8947-4489-81bc-d27e5e5da60e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:09.941426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ed468-b66f-42cf-af08-2398a14af75c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.042944 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 519/1035] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 01:06:10.044278 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0568b4-8947-4489-81bc-d27e5e5da60e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.048787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ed468-b66f-42cf-af08-2398a14af75c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:10.066658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:10.090369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff0db1-8e1b-494f-87cf-d3e34c4a3e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:10.091010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0db1-8e1b-494f-87cf-d3e34c4a3e8f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0db1-8e1b-494f-87cf-d3e34c4a3e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.091555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0db1-8e1b-494f-87cf-d3e34c4a3e8f None admin] PATH_INFO: `/v3/services/image` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.092340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff0db1-8e1b-494f-87cf-d3e34c4a3e8f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:10.093801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff0db1-8e1b-494f-87cf-d3e34c4a3e8f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-77ff0db1-8e1b-494f-87cf-d3e34c4a3e8f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:10.102850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:10.103878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:10.104743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 517/1036] 10.222.0.25 () {56 vars in 1105 bytes} [Sat Aug 6 01:06:09 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:10.109999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff0db1-8e1b-494f-87cf-d3e34c4a3e8f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.142706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:10.151995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0568b4-8947-4489-81bc-d27e5e5da60e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:10.186963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0560f-aa33-4daa-8736-65cd2836acb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:10.187741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0560f-aa33-4daa-8736-65cd2836acb4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.188146 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0560f-aa33-4daa-8736-65cd2836acb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.188518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0560f-aa33-4daa-8736-65cd2836acb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.199351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0560f-aa33-4daa-8736-65cd2836acb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:10.201008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0560f-aa33-4daa-8736-65cd2836acb4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-f3a0560f-aa33-4daa-8736-65cd2836acb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:06:10.216986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:10.218082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:10.219182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:06:10.219182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:06:10.219182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:10.219182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:10.220558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 520/1037] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:10.221566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff0db1-8e1b-494f-87cf-d3e34c4a3e8f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:10.226065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0560f-aa33-4daa-8736-65cd2836acb4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.258480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2905a6-8488-4a6c-bd09-43ac8c0b27f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:10.259107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2905a6-8488-4a6c-bd09-43ac8c0b27f6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.259615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2905a6-8488-4a6c-bd09-43ac8c0b27f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.259615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2905a6-8488-4a6c-bd09-43ac8c0b27f6 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.260359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2905a6-8488-4a6c-bd09-43ac8c0b27f6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:10.260665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2905a6-8488-4a6c-bd09-43ac8c0b27f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:10.262017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2905a6-8488-4a6c-bd09-43ac8c0b27f6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:10.271274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 518/1038] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:10.272284 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2905a6-8488-4a6c-bd09-43ac8c0b27f6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.304268 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:10.336317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0560f-aa33-4daa-8736-65cd2836acb4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:10.367037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:10.369740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396ac53-86d0-46b2-8d7f-b6fa95c729c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:10.370361 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396ac53-86d0-46b2-8d7f-b6fa95c729c5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.370639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396ac53-86d0-46b2-8d7f-b6fa95c729c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.370913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396ac53-86d0-46b2-8d7f-b6fa95c729c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.371599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396ac53-86d0-46b2-8d7f-b6fa95c729c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:10.371932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396ac53-86d0-46b2-8d7f-b6fa95c729c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:10.373295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396ac53-86d0-46b2-8d7f-b6fa95c729c5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:10.377143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2905a6-8488-4a6c-bd09-43ac8c0b27f6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:10.385644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 521/1039] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:10.391060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396ac53-86d0-46b2-8d7f-b6fa95c729c5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.418707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d46905-4eda-49db-9f39-3a367d080f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:10.419938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d46905-4eda-49db-9f39-3a367d080f13 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.420223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d46905-4eda-49db-9f39-3a367d080f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.420505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d46905-4eda-49db-9f39-3a367d080f13 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.421168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d46905-4eda-49db-9f39-3a367d080f13 None admin] RBAC: Adding query filter params (type=image) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:10.421478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d46905-4eda-49db-9f39-3a367d080f13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:10.422867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d46905-4eda-49db-9f39-3a367d080f13 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:10.427713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 519/1040] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:10.438651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d46905-4eda-49db-9f39-3a367d080f13 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.501371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396ac53-86d0-46b2-8d7f-b6fa95c729c5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:10.532481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:10.538289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e8a30-4848-4ede-a06f-9fe76cf3f3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:10.538940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e8a30-4848-4ede-a06f-9fe76cf3f3aa None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.539223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e8a30-4848-4ede-a06f-9fe76cf3f3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.539501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e8a30-4848-4ede-a06f-9fe76cf3f3aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.546306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e8a30-4848-4ede-a06f-9fe76cf3f3aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:10.547750 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e8a30-4848-4ede-a06f-9fe76cf3f3aa None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:10.550447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d46905-4eda-49db-9f39-3a367d080f13 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e09e8a30-4848-4ede-a06f-9fe76cf3f3aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:06:10.559412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:10.560340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:10.566502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:06:10.566502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:06:10.566502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:06:10.566502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:06:10.566502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:06:10.566502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:06:10.566502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:06:10.566502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:10.566502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 522/1041] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:10.567614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e8a30-4848-4ede-a06f-9fe76cf3f3aa None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.603225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab47f87-d03b-4bb3-ad62-ae710777ecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:10.603974 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab47f87-d03b-4bb3-ad62-ae710777ecbb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.604329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab47f87-d03b-4bb3-ad62-ae710777ecbb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.604674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab47f87-d03b-4bb3-ad62-ae710777ecbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.605498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab47f87-d03b-4bb3-ad62-ae710777ecbb None admin] RBAC: Adding query filter params (interface=public, service_id=0355344b19c74f05a39ff56a0ad7213f, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:10.605879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab47f87-d03b-4bb3-ad62-ae710777ecbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:10.607295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab47f87-d03b-4bb3-ad62-ae710777ecbb None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:10.612793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 520/1042] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Aug 6 01:06:10 2022] GET /identity/v3/endpoints?service_id=0355344b19c74f05a39ff56a0ad7213f&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:10.618594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab47f87-d03b-4bb3-ad62-ae710777ecbb None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.641296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:10.689438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e8a30-4848-4ede-a06f-9fe76cf3f3aa None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:10.724532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723d0b2-8583-4da4-bd55-fa57e8b0dcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:10.725158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723d0b2-8583-4da4-bd55-fa57e8b0dcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.725429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723d0b2-8583-4da4-bd55-fa57e8b0dcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.725734 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723d0b2-8583-4da4-bd55-fa57e8b0dcc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.726486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723d0b2-8583-4da4-bd55-fa57e8b0dcc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:10.726796 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723d0b2-8583-4da4-bd55-fa57e8b0dcc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:10.728286 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723d0b2-8583-4da4-bd55-fa57e8b0dcc2 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:10.728834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab47f87-d03b-4bb3-ad62-ae710777ecbb None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:10.740017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 523/1043] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:10.741210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1713b-0bc8-4ee4-947f-c343e1aa6a94 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.742026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1713b-0bc8-4ee4-947f-c343e1aa6a94 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.742300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1713b-0bc8-4ee4-947f-c343e1aa6a94 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.743700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 524/1044] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:10.746405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1713b-0bc8-4ee4-947f-c343e1aa6a94 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.764170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dae6f-9f6c-445b-a0e2-d17fc0e6a750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:10.764711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dae6f-9f6c-445b-a0e2-d17fc0e6a750 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.764989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dae6f-9f6c-445b-a0e2-d17fc0e6a750 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.765263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dae6f-9f6c-445b-a0e2-d17fc0e6a750 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.765909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dae6f-9f6c-445b-a0e2-d17fc0e6a750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:10.767260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dae6f-9f6c-445b-a0e2-d17fc0e6a750 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:10.773678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 521/1045] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Aug 6 01:06:10 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:06:10.774857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92764a30-3840-4b92-a689-f14b2e8c9f14 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.775138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92764a30-3840-4b92-a689-f14b2e8c9f14 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.775407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92764a30-3840-4b92-a689-f14b2e8c9f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.795949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-92764a30-3840-4b92-a689-f14b2e8c9f14 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:10.797186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-92764a30-3840-4b92-a689-f14b2e8c9f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:10.805674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:10.861913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1713b-0bc8-4ee4-947f-c343e1aa6a94 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:10.875954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 522/1046] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:10.899921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92764a30-3840-4b92-a689-f14b2e8c9f14 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:10.905202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782ac78-facf-4e6f-b9b6-0e1e071223b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:10.906122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782ac78-facf-4e6f-b9b6-0e1e071223b7 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:10.906339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782ac78-facf-4e6f-b9b6-0e1e071223b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:10.906669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782ac78-facf-4e6f-b9b6-0e1e071223b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:10.907524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782ac78-facf-4e6f-b9b6-0e1e071223b7 None admin] RBAC: Adding query filter params (role.id=c4fac6ed74ff415cb3e604007bcf0161, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=db1b205d165543d0a3a69a2db6af5d1c) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:10.907936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782ac78-facf-4e6f-b9b6-0e1e071223b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:10.909465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782ac78-facf-4e6f-b9b6-0e1e071223b7 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:10.916442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 525/1047] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:06:10 2022] GET /identity/v3/role_assignments?user.id=db1b205d165543d0a3a69a2db6af5d1c&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=c4fac6ed74ff415cb3e604007bcf0161 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:10.916979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:11.016958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92764a30-3840-4b92-a689-f14b2e8c9f14 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:11.060789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59183c4f-779e-4522-9113-0ea5976202e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:11.061456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183c4f-779e-4522-9113-0ea5976202e0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:11.061745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183c4f-779e-4522-9113-0ea5976202e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:11.062019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183c4f-779e-4522-9113-0ea5976202e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:11.073660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59183c4f-779e-4522-9113-0ea5976202e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-59183c4f-779e-4522-9113-0ea5976202e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:11.078598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:11.080604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:11.081533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 523/1048] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:06:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:11.086446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782ac78-facf-4e6f-b9b6-0e1e071223b7 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:11.187797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782ac78-facf-4e6f-b9b6-0e1e071223b7 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:11.204532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:11.222907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563080b-e490-4402-b69c-c41cc562b460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:11.223671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563080b-e490-4402-b69c-c41cc562b460 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:11.224069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563080b-e490-4402-b69c-c41cc562b460 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:11.224437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563080b-e490-4402-b69c-c41cc562b460 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:11.225248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563080b-e490-4402-b69c-c41cc562b460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:11.226286 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563080b-e490-4402-b69c-c41cc562b460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:11.227824 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563080b-e490-4402-b69c-c41cc562b460 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:11.236875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 526/1049] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:06:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:11.242517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59183c4f-779e-4522-9113-0ea5976202e0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:11.320144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:11.349448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59183c4f-779e-4522-9113-0ea5976202e0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:11.385409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af9bee-1cbf-4225-94c0-205746e69017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:11.386053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af9bee-1cbf-4225-94c0-205746e69017 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:11.386334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af9bee-1cbf-4225-94c0-205746e69017 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:11.386612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af9bee-1cbf-4225-94c0-205746e69017 None admin] PATH_INFO: `/v3/users/bbd1c9cbe14244dd89fa25687a9435b9` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:11.400207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af9bee-1cbf-4225-94c0-205746e69017 None admin] RBAC: Authorizing `identity:get_user(user_id=bbd1c9cbe14244dd89fa25687a9435b9)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:11.401668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af9bee-1cbf-4225-94c0-205746e69017 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:11.407970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 524/1050] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:11 2022] GET /identity/v3/users/bbd1c9cbe14244dd89fa25687a9435b9 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:11.409469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563080b-e490-4402-b69c-c41cc562b460 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:11.509221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563080b-e490-4402-b69c-c41cc562b460 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:11.542440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d074e-84cf-427a-b82a-59de303dc4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:11.543072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d074e-84cf-427a-b82a-59de303dc4dd None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:11.543350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d074e-84cf-427a-b82a-59de303dc4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:11.543618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d074e-84cf-427a-b82a-59de303dc4dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:11.595238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:11.616325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d074e-84cf-427a-b82a-59de303dc4dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:11.617782 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d074e-84cf-427a-b82a-59de303dc4dd None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:11.719597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 527/1051] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:06:11.725305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af9bee-1cbf-4225-94c0-205746e69017 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:11.821626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af9bee-1cbf-4225-94c0-205746e69017 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:11.850446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:11.855580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb6aaf-302f-4ce7-8501-72a858b8c595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:11.871940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb6aaf-302f-4ce7-8501-72a858b8c595 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:11.871940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb6aaf-302f-4ce7-8501-72a858b8c595 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:11.871940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb6aaf-302f-4ce7-8501-72a858b8c595 None admin] PATH_INFO: `/v3/projects/915a76db8fe94e4bb60b68739312acc6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:11.871940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb6aaf-302f-4ce7-8501-72a858b8c595 None admin] RBAC: Authorizing `identity:get_project(project_id=915a76db8fe94e4bb60b68739312acc6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:11.871940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb6aaf-302f-4ce7-8501-72a858b8c595 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:11.871940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 525/1052] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:11 2022] GET /identity/v3/projects/915a76db8fe94e4bb60b68739312acc6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:11.872534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d074e-84cf-427a-b82a-59de303dc4dd None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:11.955072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:11.983074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d074e-84cf-427a-b82a-59de303dc4dd None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:12.024941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c809f7c-be85-4885-aaba-c5a179d96574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:12.025731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c809f7c-be85-4885-aaba-c5a179d96574 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:12.026113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c809f7c-be85-4885-aaba-c5a179d96574 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:12.026487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c809f7c-be85-4885-aaba-c5a179d96574 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:12.027382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c809f7c-be85-4885-aaba-c5a179d96574 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=915a76db8fe94e4bb60b68739312acc6, user.id=bbd1c9cbe14244dd89fa25687a9435b9) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:12.027817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c809f7c-be85-4885-aaba-c5a179d96574 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:12.029390 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c809f7c-be85-4885-aaba-c5a179d96574 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:12.043820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 528/1053] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:06:11 2022] GET /identity/v3/role_assignments?user.id=bbd1c9cbe14244dd89fa25687a9435b9&scope.project.id=915a76db8fe94e4bb60b68739312acc6&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:12.837783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad51adf-2d08-4cf1-ae9e-aee74ca45c3c None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:12.838290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad51adf-2d08-4cf1-ae9e-aee74ca45c3c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:12.838290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad51adf-2d08-4cf1-ae9e-aee74ca45c3c None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:12.839603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 526/1054] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:12.844627 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5050b28-4976-4651-93f6-37927b227da0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:12.844915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5050b28-4976-4651-93f6-37927b227da0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:12.845190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5050b28-4976-4651-93f6-37927b227da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:12.866800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c5050b28-4976-4651-93f6-37927b227da0 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:12.868006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c5050b28-4976-4651-93f6-37927b227da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:12.933384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 529/1055] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:12.943077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767aa2d-96c8-4149-9604-189e7558daed None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:12.943366 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767aa2d-96c8-4149-9604-189e7558daed None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:12.943643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767aa2d-96c8-4149-9604-189e7558daed None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:12.945194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 527/1056] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:12.949974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa770-299b-4a26-921f-0671102a2660 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:12.950271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa770-299b-4a26-921f-0671102a2660 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:12.950556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfa770-299b-4a26-921f-0671102a2660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:12.955664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767aa2d-96c8-4149-9604-189e7558daed None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:12.969166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-34dfa770-299b-4a26-921f-0671102a2660 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:12.970362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-34dfa770-299b-4a26-921f-0671102a2660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:13.044196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 530/1057] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:13.059515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767aa2d-96c8-4149-9604-189e7558daed None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:13.060267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfa770-299b-4a26-921f-0671102a2660 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:13.062384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:13.097804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928ab73-810d-4d8c-ae57-1ca75321bc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:13.098467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928ab73-810d-4d8c-ae57-1ca75321bc75 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:13.098747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928ab73-810d-4d8c-ae57-1ca75321bc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:13.099016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928ab73-810d-4d8c-ae57-1ca75321bc75 None admin] PATH_INFO: `/v3/services/image` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:13.099639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928ab73-810d-4d8c-ae57-1ca75321bc75 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:13.101057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928ab73-810d-4d8c-ae57-1ca75321bc75 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-e928ab73-810d-4d8c-ae57-1ca75321bc75 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:13.110098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:13.111362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:13.112256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 528/1058] 10.222.0.25 () {56 vars in 1105 bytes} [Sat Aug 6 01:06:12 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:13.117645 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928ab73-810d-4d8c-ae57-1ca75321bc75 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:13.169460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfa770-299b-4a26-921f-0671102a2660 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:13.183314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:13.205463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4132c0-4d3b-40e0-9a14-50ec2532822d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:13.206123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4132c0-4d3b-40e0-9a14-50ec2532822d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:13.206396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4132c0-4d3b-40e0-9a14-50ec2532822d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:13.206671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4132c0-4d3b-40e0-9a14-50ec2532822d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:13.217721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4132c0-4d3b-40e0-9a14-50ec2532822d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:13.222225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928ab73-810d-4d8c-ae57-1ca75321bc75 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-6a4132c0-4d3b-40e0-9a14-50ec2532822d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:13.222639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:13.223553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:13.224993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 531/1059] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Aug 6 01:06:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:13.229537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4132c0-4d3b-40e0-9a14-50ec2532822d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:13.259849 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982161b-de7e-4f3b-9546-668358944cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:13.260474 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982161b-de7e-4f3b-9546-668358944cad None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:13.260755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982161b-de7e-4f3b-9546-668358944cad None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:13.261029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982161b-de7e-4f3b-9546-668358944cad None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:13.261705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982161b-de7e-4f3b-9546-668358944cad None admin] RBAC: Adding query filter params (name=image) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:13.262013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982161b-de7e-4f3b-9546-668358944cad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:13.263401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982161b-de7e-4f3b-9546-668358944cad None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:13.268835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 529/1060] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:13.274852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982161b-de7e-4f3b-9546-668358944cad None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:13.294951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:13.336603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4132c0-4d3b-40e0-9a14-50ec2532822d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:13.372415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beffd6-4d45-4029-a078-2e7907738018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:13.372960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beffd6-4d45-4029-a078-2e7907738018 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:13.373239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beffd6-4d45-4029-a078-2e7907738018 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:13.373918 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7beffd6-4d45-4029-a078-2e7907738018 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:13.374895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beffd6-4d45-4029-a078-2e7907738018 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:13.375210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beffd6-4d45-4029-a078-2e7907738018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:13.377050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7beffd6-4d45-4029-a078-2e7907738018 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:13.385607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982161b-de7e-4f3b-9546-668358944cad None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:13.391992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 532/1061] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Aug 6 01:06:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:13.393388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beffd6-4d45-4029-a078-2e7907738018 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:13.414832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:13.423464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f3585-9929-4887-bc41-988205091d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:13.424860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f3585-9929-4887-bc41-988205091d9e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:13.424860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f3585-9929-4887-bc41-988205091d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:13.424860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f3585-9929-4887-bc41-988205091d9e None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:13.425656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f3585-9929-4887-bc41-988205091d9e None admin] RBAC: Adding query filter params (type=image) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:13.425968 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f3585-9929-4887-bc41-988205091d9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:13.427345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f3585-9929-4887-bc41-988205091d9e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:13.433660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 530/1062] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:13.439842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f3585-9929-4887-bc41-988205091d9e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:13.500247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7beffd6-4d45-4029-a078-2e7907738018 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:13.522782 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:13.533558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9392c-c6a1-413f-a03d-b4bba663358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:13.534211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9392c-c6a1-413f-a03d-b4bba663358b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:13.534489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9392c-c6a1-413f-a03d-b4bba663358b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:13.534762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9392c-c6a1-413f-a03d-b4bba663358b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:13.545404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9392c-c6a1-413f-a03d-b4bba663358b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:13.546857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9392c-c6a1-413f-a03d-b4bba663358b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:13.551720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f3585-9929-4887-bc41-988205091d9e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ebf9392c-c6a1-413f-a03d-b4bba663358b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:06:13.562032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:13.562956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:13.563925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:06:13.563925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:06:13.563925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:13.563925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:13.564622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 533/1063] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:13.569394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9392c-c6a1-413f-a03d-b4bba663358b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:13.590385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b61628-5a7c-4e4d-8c78-4c2e332ba3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:13.591092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b61628-5a7c-4e4d-8c78-4c2e332ba3d2 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:13.591616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b61628-5a7c-4e4d-8c78-4c2e332ba3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:13.591942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b61628-5a7c-4e4d-8c78-4c2e332ba3d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:13.592671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b61628-5a7c-4e4d-8c78-4c2e332ba3d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:13.594287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b61628-5a7c-4e4d-8c78-4c2e332ba3d2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:13.679389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9392c-c6a1-413f-a03d-b4bba663358b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:13.703816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 531/1064] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:06:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:06:13.731448 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92db362-343c-4781-8eb1-100c9e785bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:13.732122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92db362-343c-4781-8eb1-100c9e785bee None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:13.732402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92db362-343c-4781-8eb1-100c9e785bee None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:13.732670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92db362-343c-4781-8eb1-100c9e785bee None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:13.733372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92db362-343c-4781-8eb1-100c9e785bee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:13.733675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92db362-343c-4781-8eb1-100c9e785bee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:13.735205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92db362-343c-4781-8eb1-100c9e785bee None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:13.754115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 534/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:13.755526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b61628-5a7c-4e4d-8c78-4c2e332ba3d2 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:13.810434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:13.864696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b61628-5a7c-4e4d-8c78-4c2e332ba3d2 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:13.900882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92493920-c0e8-4142-8dd0-8db92c2eadfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:13.901529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92493920-c0e8-4142-8dd0-8db92c2eadfd None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:13.901803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92493920-c0e8-4142-8dd0-8db92c2eadfd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:13.902129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92493920-c0e8-4142-8dd0-8db92c2eadfd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:13.910236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92493920-c0e8-4142-8dd0-8db92c2eadfd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:13.911690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92493920-c0e8-4142-8dd0-8db92c2eadfd None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-92493920-c0e8-4142-8dd0-8db92c2eadfd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:06:13.923817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:13.924815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:13.926166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 532/1066] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Aug 6 01:06:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:13.931283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92db362-343c-4781-8eb1-100c9e785bee None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:13.963708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:14.026868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cd944-6fac-4218-9b2c-d479080512ca None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.027214 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cd944-6fac-4218-9b2c-d479080512ca None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.027417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cd944-6fac-4218-9b2c-d479080512ca None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.028773 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 533/1067] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:06:14.033738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901dc9d-b601-4c08-817e-94f3f474a20e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.034021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901dc9d-b601-4c08-817e-94f3f474a20e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.034293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901dc9d-b601-4c08-817e-94f3f474a20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.034697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92db362-343c-4781-8eb1-100c9e785bee None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.052697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0901dc9d-b601-4c08-817e-94f3f474a20e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:14.054460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0901dc9d-b601-4c08-817e-94f3f474a20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:14.071567 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba368c-8257-4cac-a908-2000c8050d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.072224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba368c-8257-4cac-a908-2000c8050d3a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.072506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba368c-8257-4cac-a908-2000c8050d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.072788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba368c-8257-4cac-a908-2000c8050d3a None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.073585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba368c-8257-4cac-a908-2000c8050d3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:14.073884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba368c-8257-4cac-a908-2000c8050d3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:14.075339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba368c-8257-4cac-a908-2000c8050d3a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:14.097822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 535/1068] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Aug 6 01:06:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:14.099404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba368c-8257-4cac-a908-2000c8050d3a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:14.135836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 534/1069] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:14.151711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901dc9d-b601-4c08-817e-94f3f474a20e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:14.191679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:14.193089 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:14.204762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba368c-8257-4cac-a908-2000c8050d3a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.239099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ba397-8450-4b13-bd06-4386b66d6750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.239744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ba397-8450-4b13-bd06-4386b66d6750 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.240054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ba397-8450-4b13-bd06-4386b66d6750 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.240330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ba397-8450-4b13-bd06-4386b66d6750 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.250488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ba397-8450-4b13-bd06-4386b66d6750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:14.251970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ba397-8450-4b13-bd06-4386b66d6750 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:14.253074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901dc9d-b601-4c08-817e-94f3f474a20e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b80ba397-8450-4b13-bd06-4386b66d6750 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:06:14.267369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:14.268359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:14.269254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:06:14.269254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:06:14.269254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:14.269254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:14.269254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 536/1070] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:14.274557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ba397-8450-4b13-bd06-4386b66d6750 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:14.292841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69518082-adee-472a-bb11-b7d5afef815a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.293622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69518082-adee-472a-bb11-b7d5afef815a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.293907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69518082-adee-472a-bb11-b7d5afef815a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.294181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69518082-adee-472a-bb11-b7d5afef815a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.304792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69518082-adee-472a-bb11-b7d5afef815a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-69518082-adee-472a-bb11-b7d5afef815a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:14.309700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:14.310646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:14.311537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:14.315865 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 535/1071] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:14.316567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69518082-adee-472a-bb11-b7d5afef815a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:14.385153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ba397-8450-4b13-bd06-4386b66d6750 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.420536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d90e2-6d1c-4006-be62-acd7af00ca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.421163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d90e2-6d1c-4006-be62-acd7af00ca4e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.421442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d90e2-6d1c-4006-be62-acd7af00ca4e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.421721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d90e2-6d1c-4006-be62-acd7af00ca4e None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.422407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d90e2-6d1c-4006-be62-acd7af00ca4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:14.422515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:14.422709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d90e2-6d1c-4006-be62-acd7af00ca4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:14.424534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d90e2-6d1c-4006-be62-acd7af00ca4e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:14.426488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69518082-adee-472a-bb11-b7d5afef815a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.441299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 537/1072] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:14.442500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d90e2-6d1c-4006-be62-acd7af00ca4e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:14.445309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:14.469530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3aa8b-b45f-456a-8f12-8ae7fc197be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.470206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3aa8b-b45f-456a-8f12-8ae7fc197be1 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.470488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3aa8b-b45f-456a-8f12-8ae7fc197be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.470774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3aa8b-b45f-456a-8f12-8ae7fc197be1 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.471493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3aa8b-b45f-456a-8f12-8ae7fc197be1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:14.471817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3aa8b-b45f-456a-8f12-8ae7fc197be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:14.473182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3aa8b-b45f-456a-8f12-8ae7fc197be1 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:14.487428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 536/1073] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:14.488726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3aa8b-b45f-456a-8f12-8ae7fc197be1 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:14.553114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d90e2-6d1c-4006-be62-acd7af00ca4e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.590454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641dd6b-7bfa-45d0-85c6-cf8d75deeb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.591108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641dd6b-7bfa-45d0-85c6-cf8d75deeb0d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.591389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641dd6b-7bfa-45d0-85c6-cf8d75deeb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.591661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641dd6b-7bfa-45d0-85c6-cf8d75deeb0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.597757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3aa8b-b45f-456a-8f12-8ae7fc197be1 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.598338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641dd6b-7bfa-45d0-85c6-cf8d75deeb0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:14.599816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641dd6b-7bfa-45d0-85c6-cf8d75deeb0d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0641dd6b-7bfa-45d0-85c6-cf8d75deeb0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:06:14.610861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:14.611973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:14.613026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:06:14.613026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:06:14.613026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:06:14.613026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:06:14.613026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:06:14.613026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:06:14.613026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:06:14.613026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:14.613026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 538/1074] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:14.617863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641dd6b-7bfa-45d0-85c6-cf8d75deeb0d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:14.635202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56f80b-679f-4b30-a045-9743b2d58cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.635867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f80b-679f-4b30-a045-9743b2d58cef None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.636163 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f80b-679f-4b30-a045-9743b2d58cef None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.636425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f80b-679f-4b30-a045-9743b2d58cef None admin] PATH_INFO: `/v3/users/bbd1c9cbe14244dd89fa25687a9435b9` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.637588 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56f80b-679f-4b30-a045-9743b2d58cef None admin] RBAC: Authorizing `identity:get_user(user_id=bbd1c9cbe14244dd89fa25687a9435b9)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:14.639009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56f80b-679f-4b30-a045-9743b2d58cef None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:14.640764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 537/1075] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:14 2022] GET /identity/v3/users/bbd1c9cbe14244dd89fa25687a9435b9 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:14.646114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56f80b-679f-4b30-a045-9743b2d58cef None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:14.677618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:14.690201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:14.733021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641dd6b-7bfa-45d0-85c6-cf8d75deeb0d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.753722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56f80b-679f-4b30-a045-9743b2d58cef None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.768029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14873ff-9b18-4696-808b-e89ada46051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.769030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14873ff-9b18-4696-808b-e89ada46051f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.769406 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14873ff-9b18-4696-808b-e89ada46051f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.769582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14873ff-9b18-4696-808b-e89ada46051f None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.770337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14873ff-9b18-4696-808b-e89ada46051f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:14.770644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14873ff-9b18-4696-808b-e89ada46051f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:14.772176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14873ff-9b18-4696-808b-e89ada46051f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:14.790316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 539/1076] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:14.790478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac87f17-77a5-4fc0-a261-dc84bee29311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.790478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac87f17-77a5-4fc0-a261-dc84bee29311 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.790478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac87f17-77a5-4fc0-a261-dc84bee29311 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.790478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac87f17-77a5-4fc0-a261-dc84bee29311 None admin] PATH_INFO: `/v3/projects/915a76db8fe94e4bb60b68739312acc6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.791417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14873ff-9b18-4696-808b-e89ada46051f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:14.791523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac87f17-77a5-4fc0-a261-dc84bee29311 None admin] RBAC: Authorizing `identity:get_project(project_id=915a76db8fe94e4bb60b68739312acc6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:14.792951 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac87f17-77a5-4fc0-a261-dc84bee29311 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:14.794704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 538/1077] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:14 2022] GET /identity/v3/projects/915a76db8fe94e4bb60b68739312acc6 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:14.800296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac87f17-77a5-4fc0-a261-dc84bee29311 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:14.898712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:14.902087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14873ff-9b18-4696-808b-e89ada46051f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.904686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac87f17-77a5-4fc0-a261-dc84bee29311 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:14.937785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3826a7-1f9d-45d2-bcb4-5d66a055d1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.938460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3826a7-1f9d-45d2-bcb4-5d66a055d1ee None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.939007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3826a7-1f9d-45d2-bcb4-5d66a055d1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.939435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3826a7-1f9d-45d2-bcb4-5d66a055d1ee None admin] PATH_INFO: `/v3/projects/915a76db8fe94e4bb60b68739312acc6/users/bbd1c9cbe14244dd89fa25687a9435b9/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.941715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3826a7-1f9d-45d2-bcb4-5d66a055d1ee None admin] RBAC: Authorizing `identity:create_grant(project_id=915a76db8fe94e4bb60b68739312acc6, user_id=bbd1c9cbe14244dd89fa25687a9435b9, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:14.943166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3826a7-1f9d-45d2-bcb4-5d66a055d1ee None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:14.973679 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:14.984755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f77d9-bef5-41aa-9f29-6e48b2604f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:14.985370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f77d9-bef5-41aa-9f29-6e48b2604f00 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:14.985656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f77d9-bef5-41aa-9f29-6e48b2604f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:14.985934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f77d9-bef5-41aa-9f29-6e48b2604f00 None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/db1b205d165543d0a3a69a2db6af5d1c/roles/c4fac6ed74ff415cb3e604007bcf0161` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:14.988104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f77d9-bef5-41aa-9f29-6e48b2604f00 None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=db1b205d165543d0a3a69a2db6af5d1c, role_id=c4fac6ed74ff415cb3e604007bcf0161)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:14.996001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f77d9-bef5-41aa-9f29-6e48b2604f00 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:15.017128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 539/1078] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:06:14 2022] PUT /identity/v3/projects/915a76db8fe94e4bb60b68739312acc6/users/bbd1c9cbe14244dd89fa25687a9435b9/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:06:15.086638 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 540/1079] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:06:14 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/db1b205d165543d0a3a69a2db6af5d1c/roles/c4fac6ed74ff415cb3e604007bcf0161 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:06:15.771169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67d52-e86a-4275-9ff8-847a3387ade6 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:15.771513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67d52-e86a-4275-9ff8-847a3387ade6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:15.771734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd67d52-e86a-4275-9ff8-847a3387ade6 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:15.773797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 540/1080] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:15.779013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e5914-bd1c-4b0e-9e0f-3037d9e97a79 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:15.779303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e5914-bd1c-4b0e-9e0f-3037d9e97a79 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:15.779581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e5914-bd1c-4b0e-9e0f-3037d9e97a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:15.797897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-284e5914-bd1c-4b0e-9e0f-3037d9e97a79 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:15.799073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-284e5914-bd1c-4b0e-9e0f-3037d9e97a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:15.864499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 541/1081] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:06:15.886053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd67d52-e86a-4275-9ff8-847a3387ade6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:15.988650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd67d52-e86a-4275-9ff8-847a3387ade6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:16.021236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76dd58-1043-41d8-9a4d-f72902571272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:16.021849 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76dd58-1043-41d8-9a4d-f72902571272 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:16.022130 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76dd58-1043-41d8-9a4d-f72902571272 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:16.022407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76dd58-1043-41d8-9a4d-f72902571272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:16.048433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:16.094953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76dd58-1043-41d8-9a4d-f72902571272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:16.096360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76dd58-1043-41d8-9a4d-f72902571272 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:16.198389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 541/1082] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:06:16.204311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e5914-bd1c-4b0e-9e0f-3037d9e97a79 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:16.300707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e5914-bd1c-4b0e-9e0f-3037d9e97a79 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:16.313290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:16.334215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86199b14-967c-40c6-ba55-78d3d812ebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:16.334853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86199b14-967c-40c6-ba55-78d3d812ebab None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:16.335687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86199b14-967c-40c6-ba55-78d3d812ebab None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:16.335982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86199b14-967c-40c6-ba55-78d3d812ebab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:16.346459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86199b14-967c-40c6-ba55-78d3d812ebab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:16.347935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86199b14-967c-40c6-ba55-78d3d812ebab None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-86199b14-967c-40c6-ba55-78d3d812ebab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:06:16.361968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:16.366249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:16.367189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:06:16.367189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:06:16.367189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:16.367189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:16.367189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 542/1083] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:16.369057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76dd58-1043-41d8-9a4d-f72902571272 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:16.423274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:16.472082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76dd58-1043-41d8-9a4d-f72902571272 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:16.509001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfc650-5776-47b8-962d-41ee3650df05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:16.509705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfc650-5776-47b8-962d-41ee3650df05 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:16.510052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfc650-5776-47b8-962d-41ee3650df05 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:16.510387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfc650-5776-47b8-962d-41ee3650df05 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:16.511141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfc650-5776-47b8-962d-41ee3650df05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:16.511513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfc650-5776-47b8-962d-41ee3650df05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:16.512975 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfc650-5776-47b8-962d-41ee3650df05 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:16.529092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 542/1084] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:17.085520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2ae22-2707-4ecd-a881-afd103946ee0 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.085805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2ae22-2707-4ecd-a881-afd103946ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.086081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2ae22-2707-4ecd-a881-afd103946ee0 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.087456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 543/1085] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:06:17.092675 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d1ff4-adad-42a9-ae97-ccf55a46a534 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.093126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d1ff4-adad-42a9-ae97-ccf55a46a534 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.093215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d1ff4-adad-42a9-ae97-ccf55a46a534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.113521 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a80d1ff4-adad-42a9-ae97-ccf55a46a534 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:17.114707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a80d1ff4-adad-42a9-ae97-ccf55a46a534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:17.165739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac645fe-7529-40b9-aaf5-072cf220717e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.166030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac645fe-7529-40b9-aaf5-072cf220717e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.166300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac645fe-7529-40b9-aaf5-072cf220717e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.167640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 544/1086] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:06:17.174165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2f62-9dff-4cdf-9d79-7b79c8d254fb None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.174450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2f62-9dff-4cdf-9d79-7b79c8d254fb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.174730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2f62-9dff-4cdf-9d79-7b79c8d254fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.192548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 543/1087] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:17.196162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-642c2f62-9dff-4cdf-9d79-7b79c8d254fb None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:17.197706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-642c2f62-9dff-4cdf-9d79-7b79c8d254fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:17.219313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d1ff4-adad-42a9-ae97-ccf55a46a534 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:17.270879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:17.276710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 545/1088] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:06:17.301179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c2f62-9dff-4cdf-9d79-7b79c8d254fb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:17.338806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d1ff4-adad-42a9-ae97-ccf55a46a534 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:17.374852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadde88-0d9e-4284-8d54-52ec4293b40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:17.375483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadde88-0d9e-4284-8d54-52ec4293b40a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.375760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadde88-0d9e-4284-8d54-52ec4293b40a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.376062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadde88-0d9e-4284-8d54-52ec4293b40a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.386375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadde88-0d9e-4284-8d54-52ec4293b40a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-5cadde88-0d9e-4284-8d54-52ec4293b40a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:17.391315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:17.392443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:17.393580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 544/1089] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:06:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:17.398992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadde88-0d9e-4284-8d54-52ec4293b40a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:17.399523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:17.405920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c2f62-9dff-4cdf-9d79-7b79c8d254fb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:17.440945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435ef4d-5055-44ee-be46-0d49037f91c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:17.441589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435ef4d-5055-44ee-be46-0d49037f91c5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.441871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435ef4d-5055-44ee-be46-0d49037f91c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.442147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435ef4d-5055-44ee-be46-0d49037f91c5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.452046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435ef4d-5055-44ee-be46-0d49037f91c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b435ef4d-5055-44ee-be46-0d49037f91c5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:17.457091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:17.458055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:17.458910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 546/1090] 10.222.0.25 () {56 vars in 1120 bytes} [Sat Aug 6 01:06:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:17.465006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435ef4d-5055-44ee-be46-0d49037f91c5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:17.496455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadde88-0d9e-4284-8d54-52ec4293b40a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:17.503497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:17.530905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5a9d3-3318-410f-8ebd-e7ea94215d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:17.531515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5a9d3-3318-410f-8ebd-e7ea94215d6a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.531823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5a9d3-3318-410f-8ebd-e7ea94215d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.532098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5a9d3-3318-410f-8ebd-e7ea94215d6a None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.532808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5a9d3-3318-410f-8ebd-e7ea94215d6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:17.533111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5a9d3-3318-410f-8ebd-e7ea94215d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:17.534459 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5a9d3-3318-410f-8ebd-e7ea94215d6a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:17.542866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 545/1091] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:06:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:17.548767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5a9d3-3318-410f-8ebd-e7ea94215d6a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:17.566751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435ef4d-5055-44ee-be46-0d49037f91c5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:17.602161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa76ef-ad6f-440a-a6f9-7ed8ba4171da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:17.602716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa76ef-ad6f-440a-a6f9-7ed8ba4171da None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.603001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa76ef-ad6f-440a-a6f9-7ed8ba4171da None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.603277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa76ef-ad6f-440a-a6f9-7ed8ba4171da None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.604019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa76ef-ad6f-440a-a6f9-7ed8ba4171da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:17.604324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa76ef-ad6f-440a-a6f9-7ed8ba4171da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:17.605706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa76ef-ad6f-440a-a6f9-7ed8ba4171da None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:17.613283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 547/1092] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Aug 6 01:06:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:17.618870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa76ef-ad6f-440a-a6f9-7ed8ba4171da None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:17.636532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:17.647423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5a9d3-3318-410f-8ebd-e7ea94215d6a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:17.680533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e9c3b-1787-47dd-8334-f39d366c570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:17.681167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e9c3b-1787-47dd-8334-f39d366c570e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.681451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e9c3b-1787-47dd-8334-f39d366c570e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.681730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e9c3b-1787-47dd-8334-f39d366c570e None admin] PATH_INFO: `/v3/users/bbd1c9cbe14244dd89fa25687a9435b9` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.682904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e9c3b-1787-47dd-8334-f39d366c570e None admin] RBAC: Authorizing `identity:get_user(user_id=bbd1c9cbe14244dd89fa25687a9435b9)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:17.684363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e9c3b-1787-47dd-8334-f39d366c570e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:17.685961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 546/1093] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:17 2022] GET /identity/v3/users/bbd1c9cbe14244dd89fa25687a9435b9 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:17.691507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e9c3b-1787-47dd-8334-f39d366c570e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:17.717489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa76ef-ad6f-440a-a6f9-7ed8ba4171da None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:17.726015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:17.749954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c89202-5f83-4c31-92bd-a317c9ad16ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:17.750593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89202-5f83-4c31-92bd-a317c9ad16ac None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.750871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89202-5f83-4c31-92bd-a317c9ad16ac None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.751147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89202-5f83-4c31-92bd-a317c9ad16ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.761496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c89202-5f83-4c31-92bd-a317c9ad16ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:17.763081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c89202-5f83-4c31-92bd-a317c9ad16ac None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-21c89202-5f83-4c31-92bd-a317c9ad16ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:06:17.777221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:17.778292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:17.779550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:06:17.779550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:06:17.779550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:17.779550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:17.779550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 548/1094] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:17.783953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c89202-5f83-4c31-92bd-a317c9ad16ac None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:17.790911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e9c3b-1787-47dd-8334-f39d366c570e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:17.824833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457559c-e0c7-455a-8edf-1bcc0dbdda6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:17.825446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457559c-e0c7-455a-8edf-1bcc0dbdda6e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.825723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457559c-e0c7-455a-8edf-1bcc0dbdda6e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.825994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457559c-e0c7-455a-8edf-1bcc0dbdda6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.847917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:17.881387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c89202-5f83-4c31-92bd-a317c9ad16ac None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:17.891410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457559c-e0c7-455a-8edf-1bcc0dbdda6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:17.892870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457559c-e0c7-455a-8edf-1bcc0dbdda6e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:17.914324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6a27f-9efe-4f8f-98ff-cd2e2978ea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:17.914938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6a27f-9efe-4f8f-98ff-cd2e2978ea62 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:17.915225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6a27f-9efe-4f8f-98ff-cd2e2978ea62 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:17.915504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6a27f-9efe-4f8f-98ff-cd2e2978ea62 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:17.916224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6a27f-9efe-4f8f-98ff-cd2e2978ea62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:17.916534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6a27f-9efe-4f8f-98ff-cd2e2978ea62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:17.917895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6a27f-9efe-4f8f-98ff-cd2e2978ea62 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:17.925184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:17.929858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 549/1095] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:17.935346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6a27f-9efe-4f8f-98ff-cd2e2978ea62 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:17.990513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 547/1096] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 01:06:17.996265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457559c-e0c7-455a-8edf-1bcc0dbdda6e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:18.032128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6a27f-9efe-4f8f-98ff-cd2e2978ea62 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:18.057732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:18.065875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1edfc1-6dd5-41e4-8bcc-bcedfb079a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:18.066453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1edfc1-6dd5-41e4-8bcc-bcedfb079a27 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:18.066739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1edfc1-6dd5-41e4-8bcc-bcedfb079a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:18.067013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1edfc1-6dd5-41e4-8bcc-bcedfb079a27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:18.074151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1edfc1-6dd5-41e4-8bcc-bcedfb079a27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:18.075594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1edfc1-6dd5-41e4-8bcc-bcedfb079a27 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-1e1edfc1-6dd5-41e4-8bcc-bcedfb079a27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:06:18.089057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:18.090176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:18.091308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:18.091308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:18.091308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:06:18.091308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:06:18.091308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:06:18.091308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:06:18.091308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:06:18.091308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:06:18.091308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 01:06:18.091308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:18.092211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457559c-e0c7-455a-8edf-1bcc0dbdda6e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:18.093417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 550/1097] 10.222.0.25 () {56 vars in 1093 bytes} [Sat Aug 6 01:06:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:18.098995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1edfc1-6dd5-41e4-8bcc-bcedfb079a27 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:18.125562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5c2ac-cc86-4cd6-8ef5-45f9a3212ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:18.126195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5c2ac-cc86-4cd6-8ef5-45f9a3212ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:18.126476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5c2ac-cc86-4cd6-8ef5-45f9a3212ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:18.126781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5c2ac-cc86-4cd6-8ef5-45f9a3212ac7 None admin] PATH_INFO: `/v3/projects/915a76db8fe94e4bb60b68739312acc6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:18.127969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5c2ac-cc86-4cd6-8ef5-45f9a3212ac7 None admin] RBAC: Authorizing `identity:get_project(project_id=915a76db8fe94e4bb60b68739312acc6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:18.129389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5c2ac-cc86-4cd6-8ef5-45f9a3212ac7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:18.131032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 548/1098] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:17 2022] GET /identity/v3/projects/915a76db8fe94e4bb60b68739312acc6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:18.138637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5c2ac-cc86-4cd6-8ef5-45f9a3212ac7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:18.141009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:18.196217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1edfc1-6dd5-41e4-8bcc-bcedfb079a27 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:18.229585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433e882-ab35-4160-80fb-d310770954c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:18.230205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433e882-ab35-4160-80fb-d310770954c4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:18.230480 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433e882-ab35-4160-80fb-d310770954c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:18.230751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433e882-ab35-4160-80fb-d310770954c4 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:18.231535 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433e882-ab35-4160-80fb-d310770954c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:18.231884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433e882-ab35-4160-80fb-d310770954c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:18.233298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433e882-ab35-4160-80fb-d310770954c4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:18.235154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5c2ac-cc86-4cd6-8ef5-45f9a3212ac7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:18.249890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 551/1099] 10.222.0.25 () {56 vars in 1133 bytes} [Sat Aug 6 01:06:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:18.255659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433e882-ab35-4160-80fb-d310770954c4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:18.269532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382ca03-c5ef-4ff4-98f5-e4684e4cd0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:18.270149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ca03-c5ef-4ff4-98f5-e4684e4cd0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:18.270434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ca03-c5ef-4ff4-98f5-e4684e4cd0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:18.270710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ca03-c5ef-4ff4-98f5-e4684e4cd0a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:18.271516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ca03-c5ef-4ff4-98f5-e4684e4cd0a8 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=915a76db8fe94e4bb60b68739312acc6, user.id=bbd1c9cbe14244dd89fa25687a9435b9) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:18.271854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ca03-c5ef-4ff4-98f5-e4684e4cd0a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:18.273812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ca03-c5ef-4ff4-98f5-e4684e4cd0a8 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:18.279815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 549/1100] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:06:18 2022] GET /identity/v3/role_assignments?user.id=bbd1c9cbe14244dd89fa25687a9435b9&scope.project.id=915a76db8fe94e4bb60b68739312acc6&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:06:18.302618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:18.357956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433e882-ab35-4160-80fb-d310770954c4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:18.392885 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8da6b-bc1d-4c11-8d8b-ce6884d50ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:18.393437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8da6b-bc1d-4c11-8d8b-ce6884d50ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:18.393718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8da6b-bc1d-4c11-8d8b-ce6884d50ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:18.393993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8da6b-bc1d-4c11-8d8b-ce6884d50ac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:18.463654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8da6b-bc1d-4c11-8d8b-ce6884d50ac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:18.465200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8da6b-bc1d-4c11-8d8b-ce6884d50ac4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:18.508823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:18.566086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 552/1101] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 01:06:18.572514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382ca03-c5ef-4ff4-98f5-e4684e4cd0a8 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:18.641826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:18.674537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382ca03-c5ef-4ff4-98f5-e4684e4cd0a8 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:18.708831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0422c-c013-4dec-aa84-06f69d2eb261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:18.709473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0422c-c013-4dec-aa84-06f69d2eb261 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:18.709753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0422c-c013-4dec-aa84-06f69d2eb261 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:18.710043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0422c-c013-4dec-aa84-06f69d2eb261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:18.720182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0422c-c013-4dec-aa84-06f69d2eb261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:18.721613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0422c-c013-4dec-aa84-06f69d2eb261 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-54b0422c-c013-4dec-aa84-06f69d2eb261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:06:18.735256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:18.736367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:18.737325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:06:18.737325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:06:18.737325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:06:18.737325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:18.737325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 550/1102] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:18.742804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8da6b-bc1d-4c11-8d8b-ce6884d50ac4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:18.838332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8da6b-bc1d-4c11-8d8b-ce6884d50ac4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:18.870961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4ae8c-0a38-43ed-8bc7-13a7a4987b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:18.874351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4ae8c-0a38-43ed-8bc7-13a7a4987b23 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:18.874351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4ae8c-0a38-43ed-8bc7-13a7a4987b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:18.874351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4ae8c-0a38-43ed-8bc7-13a7a4987b23 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:18.874351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4ae8c-0a38-43ed-8bc7-13a7a4987b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:18.874351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4ae8c-0a38-43ed-8bc7-13a7a4987b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:18.874715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4ae8c-0a38-43ed-8bc7-13a7a4987b23 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:18.886016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 553/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:18.891744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0422c-c013-4dec-aa84-06f69d2eb261 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:18.992764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0422c-c013-4dec-aa84-06f69d2eb261 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:19.011931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:19.025104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35508a9-d9c1-43c3-87b8-b977276153fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:19.025735 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35508a9-d9c1-43c3-87b8-b977276153fe None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:19.026877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35508a9-d9c1-43c3-87b8-b977276153fe None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:19.026877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35508a9-d9c1-43c3-87b8-b977276153fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:19.032827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35508a9-d9c1-43c3-87b8-b977276153fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:19.034254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35508a9-d9c1-43c3-87b8-b977276153fe None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-a35508a9-d9c1-43c3-87b8-b977276153fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:06:19.046113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:19.047168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:19.048164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:06:19.048164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:06:19.048164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:06:19.048164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:06:19.048164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:06:19.048164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:06:19.048164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:06:19.048164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:19.048164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 551/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:06:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:06:19.052989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4ae8c-0a38-43ed-8bc7-13a7a4987b23 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:19.062639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:19.150763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4ae8c-0a38-43ed-8bc7-13a7a4987b23 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:19.183174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89a165-4997-4917-b98c-5450258ccbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:19.183808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89a165-4997-4917-b98c-5450258ccbfc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:19.184085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89a165-4997-4917-b98c-5450258ccbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:19.184356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89a165-4997-4917-b98c-5450258ccbfc None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:19.185091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89a165-4997-4917-b98c-5450258ccbfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:19.185395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89a165-4997-4917-b98c-5450258ccbfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:19.186834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89a165-4997-4917-b98c-5450258ccbfc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:19.198262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 554/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:19.204636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35508a9-d9c1-43c3-87b8-b977276153fe None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:19.299098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35508a9-d9c1-43c3-87b8-b977276153fe None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:19.331456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778e814-ad61-4e20-8711-868fd4d7f1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:19.332088 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778e814-ad61-4e20-8711-868fd4d7f1ee None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:19.332367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778e814-ad61-4e20-8711-868fd4d7f1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:19.332639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778e814-ad61-4e20-8711-868fd4d7f1ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:19.333413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778e814-ad61-4e20-8711-868fd4d7f1ee None admin] RBAC: Adding query filter params (role.id=c4fac6ed74ff415cb3e604007bcf0161, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=db1b205d165543d0a3a69a2db6af5d1c) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:19.333722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778e814-ad61-4e20-8711-868fd4d7f1ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:19.335147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778e814-ad61-4e20-8711-868fd4d7f1ee None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:19.340727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 552/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:06:19 2022] GET /identity/v3/role_assignments?user.id=db1b205d165543d0a3a69a2db6af5d1c&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=c4fac6ed74ff415cb3e604007bcf0161 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:20.345501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162b42a-aafe-4292-b508-c95b811524cb None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:20.346050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162b42a-aafe-4292-b508-c95b811524cb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:20.346050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162b42a-aafe-4292-b508-c95b811524cb None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:20.347398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 555/1107] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:20.352969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1763a36-07d6-49f9-bd09-01615662fc51 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:20.353386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1763a36-07d6-49f9-bd09-01615662fc51 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:20.353533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1763a36-07d6-49f9-bd09-01615662fc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:20.371182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e1763a36-07d6-49f9-bd09-01615662fc51 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:20.372395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e1763a36-07d6-49f9-bd09-01615662fc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:20.411041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:20.436583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 553/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:20.460893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162b42a-aafe-4292-b508-c95b811524cb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:20.523677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:20.558437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162b42a-aafe-4292-b508-c95b811524cb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:20.591584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060322b-daf8-46e4-98fd-448b610fda6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:20.592267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060322b-daf8-46e4-98fd-448b610fda6b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:20.592549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060322b-daf8-46e4-98fd-448b610fda6b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:20.592819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060322b-daf8-46e4-98fd-448b610fda6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:20.593947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060322b-daf8-46e4-98fd-448b610fda6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:20.595378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060322b-daf8-46e4-98fd-448b610fda6b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:20.597080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 556/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:20.602674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1763a36-07d6-49f9-bd09-01615662fc51 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:20.701541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1763a36-07d6-49f9-bd09-01615662fc51 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:20.734251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a4807-a2b6-49de-9e44-3358baf2ac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:20.734855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a4807-a2b6-49de-9e44-3358baf2ac3a None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:20.735136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a4807-a2b6-49de-9e44-3358baf2ac3a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:20.735410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a4807-a2b6-49de-9e44-3358baf2ac3a None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:20.736121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a4807-a2b6-49de-9e44-3358baf2ac3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:20.737530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a4807-a2b6-49de-9e44-3358baf2ac3a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:20.844863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 554/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:06:20 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:06:22.891236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd870ca-ec32-459c-baa4-f28afbde3d1b None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:22.891725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd870ca-ec32-459c-baa4-f28afbde3d1b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:22.891725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd870ca-ec32-459c-baa4-f28afbde3d1b None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:22.893113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 557/1111] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:22.898294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3820-b672-451e-a2d7-95f2f7f5fcc4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:22.898724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3820-b672-451e-a2d7-95f2f7f5fcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:22.898859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d3820-b672-451e-a2d7-95f2f7f5fcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:22.917934 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-583d3820-b672-451e-a2d7-95f2f7f5fcc4 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:22.919210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-583d3820-b672-451e-a2d7-95f2f7f5fcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:22.942524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:22.984311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 555/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:23.007577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd870ca-ec32-459c-baa4-f28afbde3d1b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:23.103857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd870ca-ec32-459c-baa4-f28afbde3d1b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:23.136068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a3a32-96b2-4926-8c19-d894cce0027a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:23.136687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a3a32-96b2-4926-8c19-d894cce0027a None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:23.136966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a3a32-96b2-4926-8c19-d894cce0027a None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:23.137237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a3a32-96b2-4926-8c19-d894cce0027a None admin] PATH_INFO: `/v3/roles/b5b241afd53c456e9a9fa3033eea74ad` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:23.138646 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a3a32-96b2-4926-8c19-d894cce0027a None admin] RBAC: Authorizing `identity:get_role(role_id=b5b241afd53c456e9a9fa3033eea74ad)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:23.140033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a3a32-96b2-4926-8c19-d894cce0027a None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:23.141401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 558/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:23 2022] GET /identity/v3/roles/b5b241afd53c456e9a9fa3033eea74ad => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:23.146984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d3820-b672-451e-a2d7-95f2f7f5fcc4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:23.243308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d3820-b672-451e-a2d7-95f2f7f5fcc4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:23.276093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1562a84-88da-422f-8414-1b00ac0b3a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:23.276723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1562a84-88da-422f-8414-1b00ac0b3a9e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:23.277008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1562a84-88da-422f-8414-1b00ac0b3a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:23.277291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1562a84-88da-422f-8414-1b00ac0b3a9e None admin] PATH_INFO: `/v3/users/941fd9381582476c9855968b6d5c06d0` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:23.290366 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1562a84-88da-422f-8414-1b00ac0b3a9e None admin] RBAC: Authorizing `identity:get_user(user_id=941fd9381582476c9855968b6d5c06d0)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:23.291804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1562a84-88da-422f-8414-1b00ac0b3a9e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:23.293406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 556/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:23 2022] GET /identity/v3/users/941fd9381582476c9855968b6d5c06d0 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:23.299073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a3a32-96b2-4926-8c19-d894cce0027a None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:23.301898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:23.393526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a3a32-96b2-4926-8c19-d894cce0027a None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:23.424350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1dcc8-6538-486a-8001-3148c8c5fa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:23.424954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1dcc8-6538-486a-8001-3148c8c5fa39 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:23.425232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1dcc8-6538-486a-8001-3148c8c5fa39 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:23.425506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1dcc8-6538-486a-8001-3148c8c5fa39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:23.488423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1dcc8-6538-486a-8001-3148c8c5fa39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:23.489849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1dcc8-6538-486a-8001-3148c8c5fa39 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:23.492112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:23.583153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 559/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:06:23.589723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1562a84-88da-422f-8414-1b00ac0b3a9e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:23.608137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:23.685359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1562a84-88da-422f-8414-1b00ac0b3a9e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:23.718234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300b1e-c6e2-466e-a760-7778666964fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:23.718859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300b1e-c6e2-466e-a760-7778666964fa None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:23.719142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300b1e-c6e2-466e-a760-7778666964fa None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:23.719420 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300b1e-c6e2-466e-a760-7778666964fa None admin] PATH_INFO: `/v3/projects/915a76db8fe94e4bb60b68739312acc6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:23.720591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71300b1e-c6e2-466e-a760-7778666964fa None admin] RBAC: Authorizing `identity:get_project(project_id=915a76db8fe94e4bb60b68739312acc6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:23.722009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71300b1e-c6e2-466e-a760-7778666964fa None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:23.723640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 557/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:23 2022] GET /identity/v3/projects/915a76db8fe94e4bb60b68739312acc6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:23.729642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1dcc8-6538-486a-8001-3148c8c5fa39 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:23.824303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1dcc8-6538-486a-8001-3148c8c5fa39 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:23.825786 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:23.857609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef74b79-873d-4df3-8681-7f805f3eb9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:23.858233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74b79-873d-4df3-8681-7f805f3eb9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:23.858500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74b79-873d-4df3-8681-7f805f3eb9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:23.858773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74b79-873d-4df3-8681-7f805f3eb9a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:23.859549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef74b79-873d-4df3-8681-7f805f3eb9a4 None admin] RBAC: Adding query filter params (role.id=b5b241afd53c456e9a9fa3033eea74ad, scope.project.id=915a76db8fe94e4bb60b68739312acc6, user.id=941fd9381582476c9855968b6d5c06d0) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:23.859888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef74b79-873d-4df3-8681-7f805f3eb9a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:23.861315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef74b79-873d-4df3-8681-7f805f3eb9a4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:23.867015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 560/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:06:23 2022] GET /identity/v3/role_assignments?user.id=941fd9381582476c9855968b6d5c06d0&scope.project.id=915a76db8fe94e4bb60b68739312acc6&role.id=b5b241afd53c456e9a9fa3033eea74ad => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:25.811141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928e446-204a-4d54-86dc-f59f876110c8 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:25.811686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928e446-204a-4d54-86dc-f59f876110c8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:25.812436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928e446-204a-4d54-86dc-f59f876110c8 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:25.814807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 558/1118] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:25.822453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2236-d0ff-4a5d-b102-a517f8c037ee None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:25.822855 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2236-d0ff-4a5d-b102-a517f8c037ee None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:25.822998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2236-d0ff-4a5d-b102-a517f8c037ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:25.841674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f3fe2236-d0ff-4a5d-b102-a517f8c037ee None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:25.842792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f3fe2236-d0ff-4a5d-b102-a517f8c037ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:25.905661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 561/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:25.927355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928e446-204a-4d54-86dc-f59f876110c8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:26.011912 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:26.027320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928e446-204a-4d54-86dc-f59f876110c8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:26.066847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2d2ea-290a-4794-ab6a-1d967cd752ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:26.067476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d2ea-290a-4794-ab6a-1d967cd752ec None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:26.067755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d2ea-290a-4794-ab6a-1d967cd752ec None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:26.068073 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d2ea-290a-4794-ab6a-1d967cd752ec None admin] PATH_INFO: `/v3/roles/b5b241afd53c456e9a9fa3033eea74ad` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:26.070014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2d2ea-290a-4794-ab6a-1d967cd752ec None admin] RBAC: Authorizing `identity:get_role(role_id=b5b241afd53c456e9a9fa3033eea74ad)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:26.071361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2d2ea-290a-4794-ab6a-1d967cd752ec None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:26.072727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 559/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:25 2022] GET /identity/v3/roles/b5b241afd53c456e9a9fa3033eea74ad => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:26.078515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe2236-d0ff-4a5d-b102-a517f8c037ee None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:26.167869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:26.173276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe2236-d0ff-4a5d-b102-a517f8c037ee None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:26.206774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb62be-458d-4ec4-9450-7eb93b5e1f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:26.207404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb62be-458d-4ec4-9450-7eb93b5e1f9c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:26.207680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb62be-458d-4ec4-9450-7eb93b5e1f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:26.207990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb62be-458d-4ec4-9450-7eb93b5e1f9c None admin] PATH_INFO: `/v3/users/941fd9381582476c9855968b6d5c06d0` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:26.209119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb62be-458d-4ec4-9450-7eb93b5e1f9c None admin] RBAC: Authorizing `identity:get_user(user_id=941fd9381582476c9855968b6d5c06d0)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:26.210540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb62be-458d-4ec4-9450-7eb93b5e1f9c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:26.212160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 562/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:26 2022] GET /identity/v3/users/941fd9381582476c9855968b6d5c06d0 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:26.217953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2d2ea-290a-4794-ab6a-1d967cd752ec None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:26.312087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2d2ea-290a-4794-ab6a-1d967cd752ec None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:26.344236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084c014-0d6c-4f32-9a85-1bd8020860f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:26.344853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084c014-0d6c-4f32-9a85-1bd8020860f6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:26.345129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084c014-0d6c-4f32-9a85-1bd8020860f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:26.345401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084c014-0d6c-4f32-9a85-1bd8020860f6 None admin] PATH_INFO: `/v3/projects/915a76db8fe94e4bb60b68739312acc6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:26.346521 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084c014-0d6c-4f32-9a85-1bd8020860f6 None admin] RBAC: Authorizing `identity:get_project(project_id=915a76db8fe94e4bb60b68739312acc6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:26.347977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084c014-0d6c-4f32-9a85-1bd8020860f6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:26.349581 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 560/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:26 2022] GET /identity/v3/projects/915a76db8fe94e4bb60b68739312acc6 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:26.355309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb62be-458d-4ec4-9450-7eb93b5e1f9c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:26.459010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb62be-458d-4ec4-9450-7eb93b5e1f9c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:26.490445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4be1a-f6de-4b1c-a5fd-58b12969a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:26.491099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4be1a-f6de-4b1c-a5fd-58b12969a602 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:26.491943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4be1a-f6de-4b1c-a5fd-58b12969a602 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:26.492217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4be1a-f6de-4b1c-a5fd-58b12969a602 None admin] PATH_INFO: `/v3/projects/915a76db8fe94e4bb60b68739312acc6/users/941fd9381582476c9855968b6d5c06d0/roles/b5b241afd53c456e9a9fa3033eea74ad` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:26.494339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4be1a-f6de-4b1c-a5fd-58b12969a602 None admin] RBAC: Authorizing `identity:create_grant(project_id=915a76db8fe94e4bb60b68739312acc6, user_id=941fd9381582476c9855968b6d5c06d0, role_id=b5b241afd53c456e9a9fa3033eea74ad)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:26.495761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4be1a-f6de-4b1c-a5fd-58b12969a602 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:26.588310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 563/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:06:26 2022] PUT /identity/v3/projects/915a76db8fe94e4bb60b68739312acc6/users/941fd9381582476c9855968b6d5c06d0/roles/b5b241afd53c456e9a9fa3033eea74ad => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:06:28.529213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a833ee-c1a5-4ed1-862f-3948854ffab7 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:28.529740 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a833ee-c1a5-4ed1-862f-3948854ffab7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:28.529740 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a833ee-c1a5-4ed1-862f-3948854ffab7 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:28.531083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 561/1124] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:28.536177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843c305-660b-4f2f-95b0-1dc849760341 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:28.536541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843c305-660b-4f2f-95b0-1dc849760341 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:28.536719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843c305-660b-4f2f-95b0-1dc849760341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:28.554830 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b843c305-660b-4f2f-95b0-1dc849760341 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:28.556038 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b843c305-660b-4f2f-95b0-1dc849760341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:28.573128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:28.620771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 564/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:28.644179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a833ee-c1a5-4ed1-862f-3948854ffab7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:28.667365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:28.744832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a833ee-c1a5-4ed1-862f-3948854ffab7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:28.777897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328fd1c-d3da-4ef6-b51d-4ee1f1207e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:28.778520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328fd1c-d3da-4ef6-b51d-4ee1f1207e42 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:28.778799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328fd1c-d3da-4ef6-b51d-4ee1f1207e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:28.779072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328fd1c-d3da-4ef6-b51d-4ee1f1207e42 None admin] PATH_INFO: `/v3/roles/b5b241afd53c456e9a9fa3033eea74ad` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:28.780561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328fd1c-d3da-4ef6-b51d-4ee1f1207e42 None admin] RBAC: Authorizing `identity:get_role(role_id=b5b241afd53c456e9a9fa3033eea74ad)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:28.781846 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328fd1c-d3da-4ef6-b51d-4ee1f1207e42 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:28.783147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 562/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:28 2022] GET /identity/v3/roles/b5b241afd53c456e9a9fa3033eea74ad => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:28.789541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843c305-660b-4f2f-95b0-1dc849760341 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:28.882465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843c305-660b-4f2f-95b0-1dc849760341 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:28.917821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c206a-e075-4bcb-9514-13a3637328f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:28.918443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c206a-e075-4bcb-9514-13a3637328f6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:28.918722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c206a-e075-4bcb-9514-13a3637328f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:28.918995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c206a-e075-4bcb-9514-13a3637328f6 None admin] PATH_INFO: `/v3/users/941fd9381582476c9855968b6d5c06d0` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:28.920168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c206a-e075-4bcb-9514-13a3637328f6 None admin] RBAC: Authorizing `identity:get_user(user_id=941fd9381582476c9855968b6d5c06d0)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:28.921591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c206a-e075-4bcb-9514-13a3637328f6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:28.923154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 565/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:28 2022] GET /identity/v3/users/941fd9381582476c9855968b6d5c06d0 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:28.928825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328fd1c-d3da-4ef6-b51d-4ee1f1207e42 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:29.019319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:29.028369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328fd1c-d3da-4ef6-b51d-4ee1f1207e42 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:29.061855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d52283-568d-43fb-940a-8249d2c64102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:29.062403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d52283-568d-43fb-940a-8249d2c64102 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:29.062685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d52283-568d-43fb-940a-8249d2c64102 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:29.062958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d52283-568d-43fb-940a-8249d2c64102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:29.126679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d52283-568d-43fb-940a-8249d2c64102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:29.128136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d52283-568d-43fb-940a-8249d2c64102 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:29.211741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:29.221287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 563/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 01:06:29.227987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c206a-e075-4bcb-9514-13a3637328f6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:29.252755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:29.324235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c206a-e075-4bcb-9514-13a3637328f6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:29.357784 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c991fe6-1ab5-47b4-a211-76446f4acf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:29.358413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c991fe6-1ab5-47b4-a211-76446f4acf9e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:29.358694 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c991fe6-1ab5-47b4-a211-76446f4acf9e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:29.358967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c991fe6-1ab5-47b4-a211-76446f4acf9e None admin] PATH_INFO: `/v3/projects/915a76db8fe94e4bb60b68739312acc6` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:29.360129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c991fe6-1ab5-47b4-a211-76446f4acf9e None admin] RBAC: Authorizing `identity:get_project(project_id=915a76db8fe94e4bb60b68739312acc6)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:29.361565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c991fe6-1ab5-47b4-a211-76446f4acf9e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:29.363210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 566/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:29 2022] GET /identity/v3/projects/915a76db8fe94e4bb60b68739312acc6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:29.369365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d52283-568d-43fb-940a-8249d2c64102 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:29.464320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d52283-568d-43fb-940a-8249d2c64102 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:29.496771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d0f6b-1de2-4924-a054-8d8f7469d723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:29.497367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d0f6b-1de2-4924-a054-8d8f7469d723 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:29.497649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d0f6b-1de2-4924-a054-8d8f7469d723 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:29.497921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d0f6b-1de2-4924-a054-8d8f7469d723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:29.498693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d0f6b-1de2-4924-a054-8d8f7469d723 None admin] RBAC: Adding query filter params (role.id=b5b241afd53c456e9a9fa3033eea74ad, scope.project.id=915a76db8fe94e4bb60b68739312acc6, user.id=941fd9381582476c9855968b6d5c06d0) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:29.499001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d0f6b-1de2-4924-a054-8d8f7469d723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:29.500434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d0f6b-1de2-4924-a054-8d8f7469d723 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:29.505939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 564/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:06:29 2022] GET /identity/v3/role_assignments?user.id=941fd9381582476c9855968b6d5c06d0&scope.project.id=915a76db8fe94e4bb60b68739312acc6&role.id=b5b241afd53c456e9a9fa3033eea74ad => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:31.489152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbbb44-1c4b-47f1-ae67-887dc8acea3f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:31.489674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbbb44-1c4b-47f1-ae67-887dc8acea3f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:31.489796 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbbb44-1c4b-47f1-ae67-887dc8acea3f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:31.491059 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 567/1131] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:31.496597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1d8ce-5681-4e77-8a3c-c1f61da89e63 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:31.496989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1d8ce-5681-4e77-8a3c-c1f61da89e63 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:31.497154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1d8ce-5681-4e77-8a3c-c1f61da89e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:31.515190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ffb1d8ce-5681-4e77-8a3c-c1f61da89e63 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:31.516394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ffb1d8ce-5681-4e77-8a3c-c1f61da89e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:31.563445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:31.581031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 565/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:31.603622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbbb44-1c4b-47f1-ae67-887dc8acea3f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:31.690695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:31.701104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbbb44-1c4b-47f1-ae67-887dc8acea3f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:31.732614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1aa9c7-c12d-47e5-b725-71da11929848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:31.733243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aa9c7-c12d-47e5-b725-71da11929848 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:31.733526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aa9c7-c12d-47e5-b725-71da11929848 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:31.733802 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aa9c7-c12d-47e5-b725-71da11929848 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:31.734936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1aa9c7-c12d-47e5-b725-71da11929848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:31.736407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1aa9c7-c12d-47e5-b725-71da11929848 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:31.737970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 568/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:31.744336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1d8ce-5681-4e77-8a3c-c1f61da89e63 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:31.838358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1d8ce-5681-4e77-8a3c-c1f61da89e63 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:31.870916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c018ae-b073-4aef-a4fc-c8887a01307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:31.871513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c018ae-b073-4aef-a4fc-c8887a01307e None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:31.871813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c018ae-b073-4aef-a4fc-c8887a01307e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:31.872093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c018ae-b073-4aef-a4fc-c8887a01307e None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:31.872778 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c018ae-b073-4aef-a4fc-c8887a01307e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:31.874180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c018ae-b073-4aef-a4fc-c8887a01307e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:31.946616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 566/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:06:31 2022] POST /identity/v3/projects => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:06:33.974746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffeefe-201b-4afd-b657-cd6369d0da6c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:33.975315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffeefe-201b-4afd-b657-cd6369d0da6c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:33.975315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffeefe-201b-4afd-b657-cd6369d0da6c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:33.977558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 569/1135] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:33.982748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c094d-1486-40b8-917e-1e1a618f80ef None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:33.983168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c094d-1486-40b8-917e-1e1a618f80ef None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:33.983303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c094d-1486-40b8-917e-1e1a618f80ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:34.001500 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5a1c094d-1486-40b8-917e-1e1a618f80ef None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:34.002659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5a1c094d-1486-40b8-917e-1e1a618f80ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:34.030204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:34.066980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 567/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:34.091861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffeefe-201b-4afd-b657-cd6369d0da6c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:34.192853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffeefe-201b-4afd-b657-cd6369d0da6c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:34.224514 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb01907-0dd2-4312-b979-a2a3ecf937ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:34.225194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb01907-0dd2-4312-b979-a2a3ecf937ae None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:34.225424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb01907-0dd2-4312-b979-a2a3ecf937ae None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:34.226203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb01907-0dd2-4312-b979-a2a3ecf937ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:34.227342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb01907-0dd2-4312-b979-a2a3ecf937ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:34.228803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb01907-0dd2-4312-b979-a2a3ecf937ae None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:34.230374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 570/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:06:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:34.236250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c094d-1486-40b8-917e-1e1a618f80ef None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:34.331330 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c094d-1486-40b8-917e-1e1a618f80ef None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:34.363483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06890cc2-ed2b-412f-87fd-525177b5f7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:34.364115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06890cc2-ed2b-412f-87fd-525177b5f7b0 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:34.364394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06890cc2-ed2b-412f-87fd-525177b5f7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:34.364662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06890cc2-ed2b-412f-87fd-525177b5f7b0 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:34.365336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06890cc2-ed2b-412f-87fd-525177b5f7b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:34.367328 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06890cc2-ed2b-412f-87fd-525177b5f7b0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:34.465437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 568/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:06:34 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:06:36.469344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e621949-e1de-49d6-8c69-779e2caf7cb3 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:36.469908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e621949-e1de-49d6-8c69-779e2caf7cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:36.469908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e621949-e1de-49d6-8c69-779e2caf7cb3 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:36.471247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 571/1139] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:36.476918 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964a340-f655-42e7-8c23-632c4aede281 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:36.477303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964a340-f655-42e7-8c23-632c4aede281 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:36.477469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964a340-f655-42e7-8c23-632c4aede281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:36.495470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5964a340-f655-42e7-8c23-632c4aede281 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:36.496668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5964a340-f655-42e7-8c23-632c4aede281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:36.512396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:36.560405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 569/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:36.586172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e621949-e1de-49d6-8c69-779e2caf7cb3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:36.613035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:36.686644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e621949-e1de-49d6-8c69-779e2caf7cb3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:36.718886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2de61b-fdaa-48b4-b4c0-250b133cfe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:36.719569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2de61b-fdaa-48b4-b4c0-250b133cfe7b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:36.719817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2de61b-fdaa-48b4-b4c0-250b133cfe7b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:36.720093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2de61b-fdaa-48b4-b4c0-250b133cfe7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:36.730226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2de61b-fdaa-48b4-b4c0-250b133cfe7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-9b2de61b-fdaa-48b4-b4c0-250b133cfe7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:36.734996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:36.736211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:36.737357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 572/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:06:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:36.748402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964a340-f655-42e7-8c23-632c4aede281 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:36.842202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964a340-f655-42e7-8c23-632c4aede281 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:36.874360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd16b7-c03d-48f4-80d6-dbca41722558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:36.874963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd16b7-c03d-48f4-80d6-dbca41722558 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:36.875243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd16b7-c03d-48f4-80d6-dbca41722558 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:36.875516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd16b7-c03d-48f4-80d6-dbca41722558 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:36.876258 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd16b7-c03d-48f4-80d6-dbca41722558 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:36.876563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd16b7-c03d-48f4-80d6-dbca41722558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:36.877914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd16b7-c03d-48f4-80d6-dbca41722558 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:36.885645 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 570/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:06:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:36.891431 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2de61b-fdaa-48b4-b4c0-250b133cfe7b None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:36.972967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:36.986874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2de61b-fdaa-48b4-b4c0-250b133cfe7b None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:37.018649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385388-7123-4631-b5d0-8d590be41104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:37.019300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385388-7123-4631-b5d0-8d590be41104 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:37.019579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385388-7123-4631-b5d0-8d590be41104 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:37.019882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61385388-7123-4631-b5d0-8d590be41104 None admin] PATH_INFO: `/v3/users/b6c6e018c0e64baa9572096e41b7e02d` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:37.032778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61385388-7123-4631-b5d0-8d590be41104 None admin] RBAC: Authorizing `identity:get_user(user_id=b6c6e018c0e64baa9572096e41b7e02d)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:37.034208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61385388-7123-4631-b5d0-8d590be41104 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:37.035803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 573/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:36 2022] GET /identity/v3/users/b6c6e018c0e64baa9572096e41b7e02d => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:37.042115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd16b7-c03d-48f4-80d6-dbca41722558 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:37.057571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:37.137760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd16b7-c03d-48f4-80d6-dbca41722558 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:37.170035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2558300-21b1-44dd-8649-4ce0f4a03714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:37.170634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558300-21b1-44dd-8649-4ce0f4a03714 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:37.170918 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558300-21b1-44dd-8649-4ce0f4a03714 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:37.171193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558300-21b1-44dd-8649-4ce0f4a03714 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:37.235242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2558300-21b1-44dd-8649-4ce0f4a03714 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:37.236633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2558300-21b1-44dd-8649-4ce0f4a03714 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:37.251364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:37.331095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 571/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:06:37.336960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385388-7123-4631-b5d0-8d590be41104 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:37.432158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61385388-7123-4631-b5d0-8d590be41104 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:37.465004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff205d3-a406-442f-8294-5ad7291d79b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:37.465632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff205d3-a406-442f-8294-5ad7291d79b4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:37.465911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff205d3-a406-442f-8294-5ad7291d79b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:37.466182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff205d3-a406-442f-8294-5ad7291d79b4 None admin] PATH_INFO: `/v3/projects/2ba82635a8a64c60aa53b9d649224cbb` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:37.467316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff205d3-a406-442f-8294-5ad7291d79b4 None admin] RBAC: Authorizing `identity:get_project(project_id=2ba82635a8a64c60aa53b9d649224cbb)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:37.468773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff205d3-a406-442f-8294-5ad7291d79b4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:37.470402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 574/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:37 2022] GET /identity/v3/projects/2ba82635a8a64c60aa53b9d649224cbb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:37.477058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2558300-21b1-44dd-8649-4ce0f4a03714 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:37.570316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2558300-21b1-44dd-8649-4ce0f4a03714 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:37.584093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:37.604042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c681b8c-a1a2-4e2e-afa5-f8a1dc744f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:37.604632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c681b8c-a1a2-4e2e-afa5-f8a1dc744f81 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:37.604909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c681b8c-a1a2-4e2e-afa5-f8a1dc744f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:37.605182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c681b8c-a1a2-4e2e-afa5-f8a1dc744f81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:37.605957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c681b8c-a1a2-4e2e-afa5-f8a1dc744f81 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=2ba82635a8a64c60aa53b9d649224cbb, user.id=b6c6e018c0e64baa9572096e41b7e02d) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:37.606267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c681b8c-a1a2-4e2e-afa5-f8a1dc744f81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:37.607662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c681b8c-a1a2-4e2e-afa5-f8a1dc744f81 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:37.612797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 572/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:06:37 2022] GET /identity/v3/role_assignments?user.id=b6c6e018c0e64baa9572096e41b7e02d&scope.project.id=2ba82635a8a64c60aa53b9d649224cbb&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:39.585851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf86559-f970-4b29-b08a-602f19da693c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:39.586296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf86559-f970-4b29-b08a-602f19da693c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:39.586496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf86559-f970-4b29-b08a-602f19da693c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:39.587756 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 575/1147] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:39.592949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5b0d0-f420-47a4-af0f-b497baba0c0a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:39.593335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5b0d0-f420-47a4-af0f-b497baba0c0a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:39.593507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5b0d0-f420-47a4-af0f-b497baba0c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:39.611613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-67a5b0d0-f420-47a4-af0f-b497baba0c0a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:39.612816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-67a5b0d0-f420-47a4-af0f-b497baba0c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:39.676672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 573/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:39.698771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf86559-f970-4b29-b08a-602f19da693c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:39.713869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:39.797216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf86559-f970-4b29-b08a-602f19da693c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:39.828707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f5556-12e4-4590-8606-fa53c57634bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:39.829333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f5556-12e4-4590-8606-fa53c57634bf None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:39.829615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f5556-12e4-4590-8606-fa53c57634bf None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:39.830132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f5556-12e4-4590-8606-fa53c57634bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:39.840341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f5556-12e4-4590-8606-fa53c57634bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-842f5556-12e4-4590-8606-fa53c57634bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:39.845085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:39.846197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:39.847465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 576/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:39.852668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5b0d0-f420-47a4-af0f-b497baba0c0a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:39.945648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5b0d0-f420-47a4-af0f-b497baba0c0a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:39.954328 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:39.978446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea7c06-84d7-45fc-9d05-9b7e612faedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:39.979056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea7c06-84d7-45fc-9d05-9b7e612faedf None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:39.979334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea7c06-84d7-45fc-9d05-9b7e612faedf None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:39.979604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea7c06-84d7-45fc-9d05-9b7e612faedf None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:39.980344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea7c06-84d7-45fc-9d05-9b7e612faedf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:39.980646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea7c06-84d7-45fc-9d05-9b7e612faedf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:39.981989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea7c06-84d7-45fc-9d05-9b7e612faedf None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:39.989586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 574/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:39.995411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f5556-12e4-4590-8606-fa53c57634bf None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:40.062734 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:40.091166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f5556-12e4-4590-8606-fa53c57634bf None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:40.124873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823854f-3042-4d43-a906-d76cbaef7d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:40.125504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823854f-3042-4d43-a906-d76cbaef7d25 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:40.125789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823854f-3042-4d43-a906-d76cbaef7d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:40.126069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823854f-3042-4d43-a906-d76cbaef7d25 None admin] PATH_INFO: `/v3/users/b6c6e018c0e64baa9572096e41b7e02d` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:40.127225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823854f-3042-4d43-a906-d76cbaef7d25 None admin] RBAC: Authorizing `identity:get_user(user_id=b6c6e018c0e64baa9572096e41b7e02d)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:40.128693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823854f-3042-4d43-a906-d76cbaef7d25 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:40.130278 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 577/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:39 2022] GET /identity/v3/users/b6c6e018c0e64baa9572096e41b7e02d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:40.136296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea7c06-84d7-45fc-9d05-9b7e612faedf None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:40.231000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea7c06-84d7-45fc-9d05-9b7e612faedf None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:40.262490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79884742-136c-47bc-8b3f-c3be1b36b24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:40.263107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79884742-136c-47bc-8b3f-c3be1b36b24c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:40.263386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79884742-136c-47bc-8b3f-c3be1b36b24c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:40.263657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79884742-136c-47bc-8b3f-c3be1b36b24c None admin] PATH_INFO: `/v3/projects/2ba82635a8a64c60aa53b9d649224cbb` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:40.264809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79884742-136c-47bc-8b3f-c3be1b36b24c None admin] RBAC: Authorizing `identity:get_project(project_id=2ba82635a8a64c60aa53b9d649224cbb)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:40.266211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79884742-136c-47bc-8b3f-c3be1b36b24c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:40.267835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 575/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:40 2022] GET /identity/v3/projects/2ba82635a8a64c60aa53b9d649224cbb => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:40.274175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823854f-3042-4d43-a906-d76cbaef7d25 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:40.368681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823854f-3042-4d43-a906-d76cbaef7d25 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:40.401801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6dee4-25c6-4c74-a839-767ceec72be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:40.402458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6dee4-25c6-4c74-a839-767ceec72be4 None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:40.402739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6dee4-25c6-4c74-a839-767ceec72be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:40.403009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6dee4-25c6-4c74-a839-767ceec72be4 None admin] PATH_INFO: `/v3/projects/2ba82635a8a64c60aa53b9d649224cbb/users/b6c6e018c0e64baa9572096e41b7e02d/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:40.404922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6dee4-25c6-4c74-a839-767ceec72be4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ba82635a8a64c60aa53b9d649224cbb, user_id=b6c6e018c0e64baa9572096e41b7e02d, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:40.406336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6dee4-25c6-4c74-a839-767ceec72be4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:40.486579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:40.491161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 578/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:06:40 2022] PUT /identity/v3/projects/2ba82635a8a64c60aa53b9d649224cbb/users/b6c6e018c0e64baa9572096e41b7e02d/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:06:42.468061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb653242-f09d-4dca-8e23-5810276f0335 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:42.468626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb653242-f09d-4dca-8e23-5810276f0335 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:42.468626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb653242-f09d-4dca-8e23-5810276f0335 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:42.469962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 576/1154] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:42.475149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7680a-6e37-407c-ab2e-2ceaaebf09bf None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:42.475520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7680a-6e37-407c-ab2e-2ceaaebf09bf None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:42.476207 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7680a-6e37-407c-ab2e-2ceaaebf09bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:42.494242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-dbc7680a-6e37-407c-ab2e-2ceaaebf09bf None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:42.495419 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-dbc7680a-6e37-407c-ab2e-2ceaaebf09bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:42.558504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 579/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:42.581588 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb653242-f09d-4dca-8e23-5810276f0335 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:42.621439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:42.685167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb653242-f09d-4dca-8e23-5810276f0335 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:42.718899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ebcb7-a062-44e0-a69d-97fdb067af86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:42.719464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ebcb7-a062-44e0-a69d-97fdb067af86 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:42.719745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ebcb7-a062-44e0-a69d-97fdb067af86 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:42.720054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ebcb7-a062-44e0-a69d-97fdb067af86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:42.730380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ebcb7-a062-44e0-a69d-97fdb067af86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bc9ebcb7-a062-44e0-a69d-97fdb067af86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:42.735207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:42.736338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:42.737396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 577/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:06:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:42.742581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7680a-6e37-407c-ab2e-2ceaaebf09bf None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:42.838221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7680a-6e37-407c-ab2e-2ceaaebf09bf None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:42.865145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:42.870562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71025c6f-aa97-438f-85fc-412ccc819a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:42.872485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71025c6f-aa97-438f-85fc-412ccc819a08 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:42.872485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71025c6f-aa97-438f-85fc-412ccc819a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:42.872485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71025c6f-aa97-438f-85fc-412ccc819a08 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:42.872485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71025c6f-aa97-438f-85fc-412ccc819a08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:42.872875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71025c6f-aa97-438f-85fc-412ccc819a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:42.874052 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71025c6f-aa97-438f-85fc-412ccc819a08 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:42.881791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 580/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:06:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:42.887832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ebcb7-a062-44e0-a69d-97fdb067af86 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:42.990933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:42.994657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ebcb7-a062-44e0-a69d-97fdb067af86 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:43.027046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd136b8-78db-4e8f-abe0-e89568525749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:43.027898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd136b8-78db-4e8f-abe0-e89568525749 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:43.028398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd136b8-78db-4e8f-abe0-e89568525749 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:43.028985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd136b8-78db-4e8f-abe0-e89568525749 None admin] PATH_INFO: `/v3/users/b6c6e018c0e64baa9572096e41b7e02d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:43.030265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd136b8-78db-4e8f-abe0-e89568525749 None admin] RBAC: Authorizing `identity:get_user(user_id=b6c6e018c0e64baa9572096e41b7e02d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:43.031680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd136b8-78db-4e8f-abe0-e89568525749 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:43.033338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 578/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:42 2022] GET /identity/v3/users/b6c6e018c0e64baa9572096e41b7e02d => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:43.039837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71025c6f-aa97-438f-85fc-412ccc819a08 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:43.136074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71025c6f-aa97-438f-85fc-412ccc819a08 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:43.168194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0a139-827d-46d9-9826-79403efaa6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:43.168802 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0a139-827d-46d9-9826-79403efaa6bc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:43.169083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0a139-827d-46d9-9826-79403efaa6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:43.169358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0a139-827d-46d9-9826-79403efaa6bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:43.224216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:43.232996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0a139-827d-46d9-9826-79403efaa6bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:43.234452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0a139-827d-46d9-9826-79403efaa6bc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:43.327258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 581/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:06:43.333445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd136b8-78db-4e8f-abe0-e89568525749 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:43.427716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd136b8-78db-4e8f-abe0-e89568525749 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:43.461450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8237aa-1264-488f-bdb4-49330e5dbdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:43.462352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8237aa-1264-488f-bdb4-49330e5dbdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:43.462743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8237aa-1264-488f-bdb4-49330e5dbdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:43.463020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8237aa-1264-488f-bdb4-49330e5dbdf7 None admin] PATH_INFO: `/v3/projects/2ba82635a8a64c60aa53b9d649224cbb` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:43.464215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8237aa-1264-488f-bdb4-49330e5dbdf7 None admin] RBAC: Authorizing `identity:get_project(project_id=2ba82635a8a64c60aa53b9d649224cbb)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:43.465635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8237aa-1264-488f-bdb4-49330e5dbdf7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:43.467260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 579/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:43 2022] GET /identity/v3/projects/2ba82635a8a64c60aa53b9d649224cbb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:43.473250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0a139-827d-46d9-9826-79403efaa6bc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:43.562679 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:43.571357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0a139-827d-46d9-9826-79403efaa6bc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:43.604175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e336f-6bd8-4845-a7b2-0634193b9cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:43.604791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e336f-6bd8-4845-a7b2-0634193b9cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:43.605073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e336f-6bd8-4845-a7b2-0634193b9cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:43.605349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e336f-6bd8-4845-a7b2-0634193b9cb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:43.606127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e336f-6bd8-4845-a7b2-0634193b9cb5 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=2ba82635a8a64c60aa53b9d649224cbb, user.id=b6c6e018c0e64baa9572096e41b7e02d) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:43.606437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e336f-6bd8-4845-a7b2-0634193b9cb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:43.607879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e336f-6bd8-4845-a7b2-0634193b9cb5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:43.613428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 582/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:06:43 2022] GET /identity/v3/role_assignments?user.id=b6c6e018c0e64baa9572096e41b7e02d&scope.project.id=2ba82635a8a64c60aa53b9d649224cbb&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:45.640530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54409c21-5bc2-4feb-a837-273464ea296f None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:45.641017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54409c21-5bc2-4feb-a837-273464ea296f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:45.641123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54409c21-5bc2-4feb-a837-273464ea296f None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:45.642426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 580/1162] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:45.647455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba736d0-879b-496a-a73e-02c995da78e2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:45.647924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba736d0-879b-496a-a73e-02c995da78e2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:45.648030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba736d0-879b-496a-a73e-02c995da78e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:45.665895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-cba736d0-879b-496a-a73e-02c995da78e2 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:45.667053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cba736d0-879b-496a-a73e-02c995da78e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:45.730445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 583/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:45.752146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54409c21-5bc2-4feb-a837-273464ea296f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:45.786204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:45.850320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54409c21-5bc2-4feb-a837-273464ea296f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:45.883405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c98150-1f01-4ce8-9afd-6bc38a27d56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:45.884041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c98150-1f01-4ce8-9afd-6bc38a27d56a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:45.884326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c98150-1f01-4ce8-9afd-6bc38a27d56a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:45.884603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c98150-1f01-4ce8-9afd-6bc38a27d56a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:45.949007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c98150-1f01-4ce8-9afd-6bc38a27d56a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:45.950445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c98150-1f01-4ce8-9afd-6bc38a27d56a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:45.982395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:46.045232 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 581/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:06:46.051173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba736d0-879b-496a-a73e-02c995da78e2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:46.130836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:46.145111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba736d0-879b-496a-a73e-02c995da78e2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:46.178718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd444f-1810-492c-bb68-396b9a60cc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:46.179353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd444f-1810-492c-bb68-396b9a60cc1d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:46.179629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd444f-1810-492c-bb68-396b9a60cc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:46.179930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd444f-1810-492c-bb68-396b9a60cc1d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:46.186254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd444f-1810-492c-bb68-396b9a60cc1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:46.187684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd444f-1810-492c-bb68-396b9a60cc1d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-4abd444f-1810-492c-bb68-396b9a60cc1d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:46.199164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:46.200300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:06:46.201279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 584/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Aug 6 01:06:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:46.206437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c98150-1f01-4ce8-9afd-6bc38a27d56a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:46.302146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c98150-1f01-4ce8-9afd-6bc38a27d56a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:46.334207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec471295-fcba-4c30-8d5f-997e6724ba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:46.334814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec471295-fcba-4c30-8d5f-997e6724ba87 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:46.335113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec471295-fcba-4c30-8d5f-997e6724ba87 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:46.335985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec471295-fcba-4c30-8d5f-997e6724ba87 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:46.336667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec471295-fcba-4c30-8d5f-997e6724ba87 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:46.336973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec471295-fcba-4c30-8d5f-997e6724ba87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:46.338326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec471295-fcba-4c30-8d5f-997e6724ba87 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:46.339729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:46.345719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 582/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Sat Aug 6 01:06:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:48.322995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1c9c5-8f39-4f8d-9d75-39db5638859c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:48.323537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1c9c5-8f39-4f8d-9d75-39db5638859c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:48.323537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1c9c5-8f39-4f8d-9d75-39db5638859c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:48.324928 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 585/1167] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:48.330082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e029f-192a-49c1-b1c6-c6626177825d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:48.330750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e029f-192a-49c1-b1c6-c6626177825d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:48.330894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e029f-192a-49c1-b1c6-c6626177825d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:48.348535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-687e029f-192a-49c1-b1c6-c6626177825d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:48.349726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-687e029f-192a-49c1-b1c6-c6626177825d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:48.412556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 583/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:48.434964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1c9c5-8f39-4f8d-9d75-39db5638859c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:48.531860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1c9c5-8f39-4f8d-9d75-39db5638859c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:48.565210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bfee5-8036-4d24-a82d-fff6ee8fcc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:48.565822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bfee5-8036-4d24-a82d-fff6ee8fcc76 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:48.566107 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bfee5-8036-4d24-a82d-fff6ee8fcc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:48.566389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bfee5-8036-4d24-a82d-fff6ee8fcc76 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:48.567104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bfee5-8036-4d24-a82d-fff6ee8fcc76 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:48.568512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bfee5-8036-4d24-a82d-fff6ee8fcc76 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:48.661030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 586/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Sat Aug 6 01:06:48 2022] POST /identity/v3/domains => generated 267 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:06:50.662059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea93d9a-70d9-4c39-b5a3-dbc9ef69f16f None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:50.662593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea93d9a-70d9-4c39-b5a3-dbc9ef69f16f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:50.662593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea93d9a-70d9-4c39-b5a3-dbc9ef69f16f None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:50.664114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 584/1170] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:06:50.669449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa78a4e-a59b-413a-9835-0f8b927ae97e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:50.669874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa78a4e-a59b-413a-9835-0f8b927ae97e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:50.669988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa78a4e-a59b-413a-9835-0f8b927ae97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:50.682788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:50.688074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3fa78a4e-a59b-413a-9835-0f8b927ae97e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:50.689265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3fa78a4e-a59b-413a-9835-0f8b927ae97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:50.753185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 587/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:50.776991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea93d9a-70d9-4c39-b5a3-dbc9ef69f16f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:50.877805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea93d9a-70d9-4c39-b5a3-dbc9ef69f16f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:50.903915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:50.911332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ceabb-db1a-4501-8780-27d2e68297c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:50.911990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ceabb-db1a-4501-8780-27d2e68297c9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:50.912273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ceabb-db1a-4501-8780-27d2e68297c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:50.912548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ceabb-db1a-4501-8780-27d2e68297c9 None admin] PATH_INFO: `/v3/domains/7db99fc70201458abe3b651a79db79d0` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:50.922565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ceabb-db1a-4501-8780-27d2e68297c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7db99fc70201458abe3b651a79db79d0)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:50.924034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ceabb-db1a-4501-8780-27d2e68297c9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:50.925693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 585/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Sat Aug 6 01:06:50 2022] GET /identity/v3/domains/7db99fc70201458abe3b651a79db79d0 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:50.932289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa78a4e-a59b-413a-9835-0f8b927ae97e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:51.027787 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa78a4e-a59b-413a-9835-0f8b927ae97e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:51.060008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893c70a-1f01-4355-9078-e2d673459c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:51.060616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893c70a-1f01-4355-9078-e2d673459c02 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:51.060898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893c70a-1f01-4355-9078-e2d673459c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:51.061171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893c70a-1f01-4355-9078-e2d673459c02 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:51.061852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893c70a-1f01-4355-9078-e2d673459c02 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:51.063271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893c70a-1f01-4355-9078-e2d673459c02 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:51.068432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:51.152621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 588/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:06:50 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:06:53.182246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea449e-7b49-45af-b73e-95fd2a916b8a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:53.182774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea449e-7b49-45af-b73e-95fd2a916b8a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:53.182774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea449e-7b49-45af-b73e-95fd2a916b8a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:53.184190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 586/1174] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:53.189221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a86ea6-d74e-48a0-9add-1c7f6c7dea4d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:53.189648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a86ea6-d74e-48a0-9add-1c7f6c7dea4d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:53.189752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a86ea6-d74e-48a0-9add-1c7f6c7dea4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:53.208067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-36a86ea6-d74e-48a0-9add-1c7f6c7dea4d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:53.209242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-36a86ea6-d74e-48a0-9add-1c7f6c7dea4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:53.275318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 589/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:53.299345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea449e-7b49-45af-b73e-95fd2a916b8a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:53.397120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea449e-7b49-45af-b73e-95fd2a916b8a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:53.429696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861089be-6ffc-46a8-8179-b9fd0d1509cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:53.430322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861089be-6ffc-46a8-8179-b9fd0d1509cc None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:53.430604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861089be-6ffc-46a8-8179-b9fd0d1509cc None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:53.430876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861089be-6ffc-46a8-8179-b9fd0d1509cc None admin] PATH_INFO: `/v3/domains/7db99fc70201458abe3b651a79db79d0` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:53.432062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861089be-6ffc-46a8-8179-b9fd0d1509cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=7db99fc70201458abe3b651a79db79d0)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:53.433532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861089be-6ffc-46a8-8179-b9fd0d1509cc None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:53.435035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 587/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Sat Aug 6 01:06:53 2022] GET /identity/v3/domains/7db99fc70201458abe3b651a79db79d0 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:53.440621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a86ea6-d74e-48a0-9add-1c7f6c7dea4d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:53.534513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a86ea6-d74e-48a0-9add-1c7f6c7dea4d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:53.551183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:53.567243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598cd3c-a4a2-4b21-97e7-b1f5924e08ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:53.571021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598cd3c-a4a2-4b21-97e7-b1f5924e08ed None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:53.571021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598cd3c-a4a2-4b21-97e7-b1f5924e08ed None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:53.571021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598cd3c-a4a2-4b21-97e7-b1f5924e08ed None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:53.571021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598cd3c-a4a2-4b21-97e7-b1f5924e08ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:53.571021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598cd3c-a4a2-4b21-97e7-b1f5924e08ed None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:53.666106 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 590/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:06:53 2022] POST /identity/v3/users => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:06:55.678682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8df5a1-91a1-46fa-83fd-775f9b9f97a7 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:55.679801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8df5a1-91a1-46fa-83fd-775f9b9f97a7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:55.679801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8df5a1-91a1-46fa-83fd-775f9b9f97a7 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:55.681153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 588/1178] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:55.686206 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb4419-a439-4ddf-afa3-685c1deb6429 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:55.686612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb4419-a439-4ddf-afa3-685c1deb6429 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:55.686756 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb4419-a439-4ddf-afa3-685c1deb6429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:55.706158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b5eb4419-a439-4ddf-afa3-685c1deb6429 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:55.707367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b5eb4419-a439-4ddf-afa3-685c1deb6429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:55.770194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 591/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:06:55.792902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8df5a1-91a1-46fa-83fd-775f9b9f97a7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:55.848263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:55.890960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8df5a1-91a1-46fa-83fd-775f9b9f97a7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:55.923636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d5e36-47a1-41e4-899f-7fb849376164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:55.924350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d5e36-47a1-41e4-899f-7fb849376164 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:55.924632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d5e36-47a1-41e4-899f-7fb849376164 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:55.924908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d5e36-47a1-41e4-899f-7fb849376164 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:55.935248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d5e36-47a1-41e4-899f-7fb849376164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-7a1d5e36-47a1-41e4-899f-7fb849376164 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:55.940117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:55.941050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:55.941959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 589/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:06:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:55.947299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb4419-a439-4ddf-afa3-685c1deb6429 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:56.042399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb4419-a439-4ddf-afa3-685c1deb6429 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:56.044619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:56.074113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d48f90-b5fc-46b4-ab67-11705c946e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:56.074739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d48f90-b5fc-46b4-ab67-11705c946e07 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:56.075002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d48f90-b5fc-46b4-ab67-11705c946e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:56.075272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d48f90-b5fc-46b4-ab67-11705c946e07 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:56.076002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d48f90-b5fc-46b4-ab67-11705c946e07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:56.076316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d48f90-b5fc-46b4-ab67-11705c946e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:56.078160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d48f90-b5fc-46b4-ab67-11705c946e07 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:56.086316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 592/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:06:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:56.096139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d5e36-47a1-41e4-899f-7fb849376164 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:56.188887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d5e36-47a1-41e4-899f-7fb849376164 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:56.204948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:56.222081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dd42f-8306-4fc0-a3e0-c3acffaef631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:56.222707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dd42f-8306-4fc0-a3e0-c3acffaef631 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:56.222991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dd42f-8306-4fc0-a3e0-c3acffaef631 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:56.223269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dd42f-8306-4fc0-a3e0-c3acffaef631 None admin] PATH_INFO: `/v3/users/bceb377ff60046fba16d1bff223c7390` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:56.236582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dd42f-8306-4fc0-a3e0-c3acffaef631 None admin] RBAC: Authorizing `identity:get_user(user_id=bceb377ff60046fba16d1bff223c7390)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:56.238733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dd42f-8306-4fc0-a3e0-c3acffaef631 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:56.240377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 590/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:56 2022] GET /identity/v3/users/bceb377ff60046fba16d1bff223c7390 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:56.246949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d48f90-b5fc-46b4-ab67-11705c946e07 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:56.359739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d48f90-b5fc-46b4-ab67-11705c946e07 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:56.400460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca35ba9-c6bd-4e47-aca7-b72cf0b8dcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:56.401347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca35ba9-c6bd-4e47-aca7-b72cf0b8dcda None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:56.401771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca35ba9-c6bd-4e47-aca7-b72cf0b8dcda None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:56.402180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca35ba9-c6bd-4e47-aca7-b72cf0b8dcda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:56.434854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:56.467499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca35ba9-c6bd-4e47-aca7-b72cf0b8dcda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:56.468965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca35ba9-c6bd-4e47-aca7-b72cf0b8dcda None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:56.560434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 593/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:06:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:06:56.566323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dd42f-8306-4fc0-a3e0-c3acffaef631 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:56.662637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dd42f-8306-4fc0-a3e0-c3acffaef631 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:56.706753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646570ed-b0eb-47b1-b182-d2c52bb39df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:56.707454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646570ed-b0eb-47b1-b182-d2c52bb39df5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:56.707679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646570ed-b0eb-47b1-b182-d2c52bb39df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:56.707991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646570ed-b0eb-47b1-b182-d2c52bb39df5 None admin] PATH_INFO: `/v3/projects/1f0841d335a34a579dd982934dba4d7c` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:56.710781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646570ed-b0eb-47b1-b182-d2c52bb39df5 None admin] RBAC: Authorizing `identity:get_project(project_id=1f0841d335a34a579dd982934dba4d7c)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:56.711362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646570ed-b0eb-47b1-b182-d2c52bb39df5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:56.714953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 591/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:56 2022] GET /identity/v3/projects/1f0841d335a34a579dd982934dba4d7c => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:56.719273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca35ba9-c6bd-4e47-aca7-b72cf0b8dcda None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:56.802447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:56.838150 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca35ba9-c6bd-4e47-aca7-b72cf0b8dcda None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:56.869988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68d5c3-3a6c-4414-9a48-ff66240437fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:56.870597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68d5c3-3a6c-4414-9a48-ff66240437fd None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:56.870879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68d5c3-3a6c-4414-9a48-ff66240437fd None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:56.871157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68d5c3-3a6c-4414-9a48-ff66240437fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:56.872094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68d5c3-3a6c-4414-9a48-ff66240437fd None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=1f0841d335a34a579dd982934dba4d7c, user.id=bceb377ff60046fba16d1bff223c7390) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:56.872409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68d5c3-3a6c-4414-9a48-ff66240437fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:56.873843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68d5c3-3a6c-4414-9a48-ff66240437fd None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:56.879077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 594/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:06:56 2022] GET /identity/v3/role_assignments?user.id=bceb377ff60046fba16d1bff223c7390&scope.project.id=1f0841d335a34a579dd982934dba4d7c&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:58.938434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd087305-e2ba-47aa-8254-04c36a1e96d6 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:58.938944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd087305-e2ba-47aa-8254-04c36a1e96d6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:58.938944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd087305-e2ba-47aa-8254-04c36a1e96d6 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:58.940265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 592/1186] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:06:58.945438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dadbe7-0111-438c-9b46-308ecb407b27 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:58.945842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dadbe7-0111-438c-9b46-308ecb407b27 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:58.945991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dadbe7-0111-438c-9b46-308ecb407b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:58.963843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c1dadbe7-0111-438c-9b46-308ecb407b27 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:06:58.965022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c1dadbe7-0111-438c-9b46-308ecb407b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:06:59.028505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 595/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:06:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:06:59.050375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd087305-e2ba-47aa-8254-04c36a1e96d6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:59.093147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:59.147078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd087305-e2ba-47aa-8254-04c36a1e96d6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:59.179822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166384-2058-4c15-99a3-190dd4832292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:59.180442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166384-2058-4c15-99a3-190dd4832292 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:59.180719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166384-2058-4c15-99a3-190dd4832292 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:59.180985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166384-2058-4c15-99a3-190dd4832292 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:59.190616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3166384-2058-4c15-99a3-190dd4832292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-d3166384-2058-4c15-99a3-190dd4832292 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:06:59.195407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:06:59.196523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:06:59.197426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 593/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:06:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:06:59.202487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dadbe7-0111-438c-9b46-308ecb407b27 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:59.258692 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:59.297634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dadbe7-0111-438c-9b46-308ecb407b27 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:59.329117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96e6eb-0dbe-4a96-99a5-4e48deb542a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:59.329726 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96e6eb-0dbe-4a96-99a5-4e48deb542a0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:59.330003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96e6eb-0dbe-4a96-99a5-4e48deb542a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:59.330279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96e6eb-0dbe-4a96-99a5-4e48deb542a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:59.331243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96e6eb-0dbe-4a96-99a5-4e48deb542a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:06:59.331543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96e6eb-0dbe-4a96-99a5-4e48deb542a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:59.332935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b96e6eb-0dbe-4a96-99a5-4e48deb542a0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:59.341090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 596/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:06:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:59.346742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166384-2058-4c15-99a3-190dd4832292 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:59.442281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166384-2058-4c15-99a3-190dd4832292 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:59.443676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:59.482630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bbe74-259a-4c61-9829-fc4f34cc6cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:59.483257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bbe74-259a-4c61-9829-fc4f34cc6cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:59.483539 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bbe74-259a-4c61-9829-fc4f34cc6cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:59.483835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bbe74-259a-4c61-9829-fc4f34cc6cc5 None admin] PATH_INFO: `/v3/users/bceb377ff60046fba16d1bff223c7390` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:59.484982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bbe74-259a-4c61-9829-fc4f34cc6cc5 None admin] RBAC: Authorizing `identity:get_user(user_id=bceb377ff60046fba16d1bff223c7390)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:59.486392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bbe74-259a-4c61-9829-fc4f34cc6cc5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:59.488037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 594/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:06:59 2022] GET /identity/v3/users/bceb377ff60046fba16d1bff223c7390 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:59.493743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96e6eb-0dbe-4a96-99a5-4e48deb542a0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:59.588670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96e6eb-0dbe-4a96-99a5-4e48deb542a0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:59.609637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:06:59.620524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6ce7c-e53b-470b-bd85-63f35c2a4a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:59.621155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6ce7c-e53b-470b-bd85-63f35c2a4a55 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:59.630655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6ce7c-e53b-470b-bd85-63f35c2a4a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:59.630655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6ce7c-e53b-470b-bd85-63f35c2a4a55 None admin] PATH_INFO: `/v3/projects/1f0841d335a34a579dd982934dba4d7c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:59.630655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6ce7c-e53b-470b-bd85-63f35c2a4a55 None admin] RBAC: Authorizing `identity:get_project(project_id=1f0841d335a34a579dd982934dba4d7c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:59.630655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6ce7c-e53b-470b-bd85-63f35c2a4a55 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:59.630655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 597/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:06:59 2022] GET /identity/v3/projects/1f0841d335a34a579dd982934dba4d7c => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:06:59.631668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bbe74-259a-4c61-9829-fc4f34cc6cc5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:06:59.726494 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bbe74-259a-4c61-9829-fc4f34cc6cc5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:06:59.759431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1bb40-cc77-4d86-9705-c1039073fa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:06:59.760112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1bb40-cc77-4d86-9705-c1039073fa22 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:06:59.760391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1bb40-cc77-4d86-9705-c1039073fa22 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:06:59.760665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1bb40-cc77-4d86-9705-c1039073fa22 None admin] PATH_INFO: `/v3/projects/1f0841d335a34a579dd982934dba4d7c/users/bceb377ff60046fba16d1bff223c7390/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:06:59.762568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1bb40-cc77-4d86-9705-c1039073fa22 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f0841d335a34a579dd982934dba4d7c, user_id=bceb377ff60046fba16d1bff223c7390, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:06:59.764006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1bb40-cc77-4d86-9705-c1039073fa22 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:06:59.831743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 595/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:06:59 2022] PUT /identity/v3/projects/1f0841d335a34a579dd982934dba4d7c/users/bceb377ff60046fba16d1bff223c7390/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:07:01.789988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5264d39-2cc7-49ec-a636-9f025a325867 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:07:01.790495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5264d39-2cc7-49ec-a636-9f025a325867 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:07:01.790588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5264d39-2cc7-49ec-a636-9f025a325867 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:07:01.791912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 598/1193] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:07:01.797085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1573-a329-45a6-bfc3-0c7a0b85d67e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:07:01.797506 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1573-a329-45a6-bfc3-0c7a0b85d67e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:07:01.797640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1573-a329-45a6-bfc3-0c7a0b85d67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:07:01.815745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-eb4c1573-a329-45a6-bfc3-0c7a0b85d67e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:07:01.816942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-eb4c1573-a329-45a6-bfc3-0c7a0b85d67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:07:01.839373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:07:01.881444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 596/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:07:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:07:01.904017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5264d39-2cc7-49ec-a636-9f025a325867 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:07:02.001137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5264d39-2cc7-49ec-a636-9f025a325867 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:07:02.032516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb1b2b-7000-473b-a15a-af3f572e22a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:07:02.033141 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb1b2b-7000-473b-a15a-af3f572e22a5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:07:02.033423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb1b2b-7000-473b-a15a-af3f572e22a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:07:02.033697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb1b2b-7000-473b-a15a-af3f572e22a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:07:02.043857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb1b2b-7000-473b-a15a-af3f572e22a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-6ccb1b2b-7000-473b-a15a-af3f572e22a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:07:02.048657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:07:02.049753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:07:02.050969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 599/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:07:02.056128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c1573-a329-45a6-bfc3-0c7a0b85d67e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:07:02.149381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c1573-a329-45a6-bfc3-0c7a0b85d67e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:07:02.182382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac4345-d553-42c9-82c5-ad1a6591ea96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:07:02.182986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac4345-d553-42c9-82c5-ad1a6591ea96 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:07:02.183268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac4345-d553-42c9-82c5-ad1a6591ea96 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:07:02.183573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac4345-d553-42c9-82c5-ad1a6591ea96 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:07:02.184318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac4345-d553-42c9-82c5-ad1a6591ea96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:07:02.184625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac4345-d553-42c9-82c5-ad1a6591ea96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:07:02.185979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac4345-d553-42c9-82c5-ad1a6591ea96 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:07:02.193656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 597/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:07:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:07:02.199099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb1b2b-7000-473b-a15a-af3f572e22a5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:07:02.247914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:07:02.295651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb1b2b-7000-473b-a15a-af3f572e22a5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:07:02.327154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95c58e-8b5f-4669-beeb-5ab744e15f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:07:02.327819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95c58e-8b5f-4669-beeb-5ab744e15f65 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:07:02.328082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95c58e-8b5f-4669-beeb-5ab744e15f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:07:02.328354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95c58e-8b5f-4669-beeb-5ab744e15f65 None admin] PATH_INFO: `/v3/users/bceb377ff60046fba16d1bff223c7390` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:07:02.329482 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95c58e-8b5f-4669-beeb-5ab744e15f65 None admin] RBAC: Authorizing `identity:get_user(user_id=bceb377ff60046fba16d1bff223c7390)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:07:02.331161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95c58e-8b5f-4669-beeb-5ab744e15f65 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:07:02.332777 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 600/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Sat Aug 6 01:07:02 2022] GET /identity/v3/users/bceb377ff60046fba16d1bff223c7390 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:07:02.338998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac4345-d553-42c9-82c5-ad1a6591ea96 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:07:02.368187 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:07:02.441327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac4345-d553-42c9-82c5-ad1a6591ea96 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:07:02.473958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704ecf1-0e1a-495e-b0f1-321ec2380bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:07:02.474567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704ecf1-0e1a-495e-b0f1-321ec2380bfd None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:07:02.474853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704ecf1-0e1a-495e-b0f1-321ec2380bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:07:02.475132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704ecf1-0e1a-495e-b0f1-321ec2380bfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:07:02.539502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704ecf1-0e1a-495e-b0f1-321ec2380bfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:07:02.540965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704ecf1-0e1a-495e-b0f1-321ec2380bfd None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:07:02.562595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:07:02.636050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 598/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:07:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:07:02.641841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95c58e-8b5f-4669-beeb-5ab744e15f65 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:07:02.735858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95c58e-8b5f-4669-beeb-5ab744e15f65 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:07:02.752460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:07:02.768873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82462720-8a6a-4117-8ce8-c01156d30dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:07:02.769506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82462720-8a6a-4117-8ce8-c01156d30dac None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:07:02.769912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82462720-8a6a-4117-8ce8-c01156d30dac None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:07:02.770188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82462720-8a6a-4117-8ce8-c01156d30dac None admin] PATH_INFO: `/v3/projects/1f0841d335a34a579dd982934dba4d7c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:07:02.771321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82462720-8a6a-4117-8ce8-c01156d30dac None admin] RBAC: Authorizing `identity:get_project(project_id=1f0841d335a34a579dd982934dba4d7c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:07:02.773345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82462720-8a6a-4117-8ce8-c01156d30dac None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:07:02.775024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 601/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Sat Aug 6 01:07:02 2022] GET /identity/v3/projects/1f0841d335a34a579dd982934dba4d7c => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:07:02.781137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704ecf1-0e1a-495e-b0f1-321ec2380bfd None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:07:02.878842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704ecf1-0e1a-495e-b0f1-321ec2380bfd None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:07:02.902503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:07:02.913573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d09bff-2d08-4625-a043-60992755c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:07:02.914179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d09bff-2d08-4625-a043-60992755c479 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:07:02.914457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d09bff-2d08-4625-a043-60992755c479 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:07:02.914730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d09bff-2d08-4625-a043-60992755c479 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:07:02.915510 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d09bff-2d08-4625-a043-60992755c479 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=1f0841d335a34a579dd982934dba4d7c, user.id=bceb377ff60046fba16d1bff223c7390) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:07:02.915900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d09bff-2d08-4625-a043-60992755c479 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:07:02.917309 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d09bff-2d08-4625-a043-60992755c479 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:07:02.923030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 599/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:07:02 2022] GET /identity/v3/role_assignments?user.id=bceb377ff60046fba16d1bff223c7390&scope.project.id=1f0841d335a34a579dd982934dba4d7c&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:07:12.782787 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:07:12.929299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:14.067385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceb8b3-8433-4eea-acf8-940103a9b6b4 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:14.068074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceb8b3-8433-4eea-acf8-940103a9b6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:14.068074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceb8b3-8433-4eea-acf8-940103a9b6b4 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:14.069413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 602/1201] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:08:14.074964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f5379-4b25-4040-b24a-119f68b51a97 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:14.075435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f5379-4b25-4040-b24a-119f68b51a97 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:14.075534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f5379-4b25-4040-b24a-119f68b51a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:14.094247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-356f5379-4b25-4040-b24a-119f68b51a97 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:14.095460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-356f5379-4b25-4040-b24a-119f68b51a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:14.165074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 600/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:08:14.188916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aceb8b3-8433-4eea-acf8-940103a9b6b4 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:14.288425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aceb8b3-8433-4eea-acf8-940103a9b6b4 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:14.321853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc99979-8924-4c76-b26a-e18d2ba63589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:14.322423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc99979-8924-4c76-b26a-e18d2ba63589 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:14.322701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc99979-8924-4c76-b26a-e18d2ba63589 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:14.322972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc99979-8924-4c76-b26a-e18d2ba63589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:14.332664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc99979-8924-4c76-b26a-e18d2ba63589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:14.334366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc99979-8924-4c76-b26a-e18d2ba63589 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ecc99979-8924-4c76-b26a-e18d2ba63589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:14.348343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:14.349844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:14.351354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:14.351354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:14.351354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:14.351354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:14.351354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 603/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:08:14.355394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f5379-4b25-4040-b24a-119f68b51a97 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:14.450645 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f5379-4b25-4040-b24a-119f68b51a97 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:14.477842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:14.483440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfb67e-293a-4e44-ba42-a111f38af281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:14.484082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfb67e-293a-4e44-ba42-a111f38af281 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:14.484361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfb67e-293a-4e44-ba42-a111f38af281 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:14.484635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfb67e-293a-4e44-ba42-a111f38af281 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:14.485327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfb67e-293a-4e44-ba42-a111f38af281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:14.485631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfb67e-293a-4e44-ba42-a111f38af281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:14.486986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfb67e-293a-4e44-ba42-a111f38af281 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:14.498283 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 601/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:14.504076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc99979-8924-4c76-b26a-e18d2ba63589 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:14.571824 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:14.619656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc99979-8924-4c76-b26a-e18d2ba63589 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:14.651120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6cb92-bb87-4d3d-8221-4d810c5b3152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:14.651741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6cb92-bb87-4d3d-8221-4d810c5b3152 None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:14.652048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6cb92-bb87-4d3d-8221-4d810c5b3152 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:14.652321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6cb92-bb87-4d3d-8221-4d810c5b3152 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:14.653006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6cb92-bb87-4d3d-8221-4d810c5b3152 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:14.654437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6cb92-bb87-4d3d-8221-4d810c5b3152 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:14.769564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 604/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Sat Aug 6 01:08:14 2022] POST /identity/v3/users => generated 251 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:08:16.809146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3bab0-400c-4153-9c46-3a90f2f81857 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:16.809696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3bab0-400c-4153-9c46-3a90f2f81857 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:16.809696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3bab0-400c-4153-9c46-3a90f2f81857 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:16.811947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 602/1206] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:08:16.822674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148264f-c1d0-4c88-a83a-0c54280708ef None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:16.822674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148264f-c1d0-4c88-a83a-0c54280708ef None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:16.822674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148264f-c1d0-4c88-a83a-0c54280708ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:16.844142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b148264f-c1d0-4c88-a83a-0c54280708ef None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:16.845815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b148264f-c1d0-4c88-a83a-0c54280708ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:16.919569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 605/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:08:16.942635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3bab0-400c-4153-9c46-3a90f2f81857 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:17.041470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3bab0-400c-4153-9c46-3a90f2f81857 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:17.074435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013cd64-b9c5-4e98-9b8e-a67065492b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:17.075068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013cd64-b9c5-4e98-9b8e-a67065492b17 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:17.075351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013cd64-b9c5-4e98-9b8e-a67065492b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:17.075628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013cd64-b9c5-4e98-9b8e-a67065492b17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:17.086502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013cd64-b9c5-4e98-9b8e-a67065492b17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-2013cd64-b9c5-4e98-9b8e-a67065492b17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:08:17.091321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:17.092457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:17.093415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 603/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:08:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:17.098575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148264f-c1d0-4c88-a83a-0c54280708ef None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:17.140994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:17.200019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148264f-c1d0-4c88-a83a-0c54280708ef None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:17.232314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27c3aa-01dc-4aaa-b7a8-8e6cdf1330ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:17.232930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27c3aa-01dc-4aaa-b7a8-8e6cdf1330ed None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:17.233209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27c3aa-01dc-4aaa-b7a8-8e6cdf1330ed None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:17.233481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27c3aa-01dc-4aaa-b7a8-8e6cdf1330ed None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:17.234187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27c3aa-01dc-4aaa-b7a8-8e6cdf1330ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:17.234491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27c3aa-01dc-4aaa-b7a8-8e6cdf1330ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:17.235903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27c3aa-01dc-4aaa-b7a8-8e6cdf1330ed None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:17.244194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 606/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:08:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:17.249943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013cd64-b9c5-4e98-9b8e-a67065492b17 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:17.302529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:17.353568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013cd64-b9c5-4e98-9b8e-a67065492b17 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:17.389427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0ed63-8735-48ab-9a68-274dd2aa387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:17.389427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ed63-8735-48ab-9a68-274dd2aa387c None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:17.389427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ed63-8735-48ab-9a68-274dd2aa387c None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:17.389427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0ed63-8735-48ab-9a68-274dd2aa387c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:17.401029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0ed63-8735-48ab-9a68-274dd2aa387c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:17.402530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0ed63-8735-48ab-9a68-274dd2aa387c None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-22b0ed63-8735-48ab-9a68-274dd2aa387c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:17.417716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:17.418644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:17.419636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:17.419636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:17.419636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:17.419636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:17.419636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 604/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:17.431497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27c3aa-01dc-4aaa-b7a8-8e6cdf1330ed None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:17.524562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:17.527179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27c3aa-01dc-4aaa-b7a8-8e6cdf1330ed None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:17.561960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87f5bd-5ed2-41ff-b706-ec2bdec82ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:17.562586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f5bd-5ed2-41ff-b706-ec2bdec82ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:17.562876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f5bd-5ed2-41ff-b706-ec2bdec82ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:17.563144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87f5bd-5ed2-41ff-b706-ec2bdec82ef6 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:17.563918 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87f5bd-5ed2-41ff-b706-ec2bdec82ef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:17.564332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87f5bd-5ed2-41ff-b706-ec2bdec82ef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:17.565605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87f5bd-5ed2-41ff-b706-ec2bdec82ef6 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:17.576650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 607/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:17.582418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0ed63-8735-48ab-9a68-274dd2aa387c None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:17.680983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0ed63-8735-48ab-9a68-274dd2aa387c None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:17.682401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:17.716640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77424c8-f362-4f38-ba4d-c64deb24c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:17.717272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77424c8-f362-4f38-ba4d-c64deb24c995 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:17.717549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77424c8-f362-4f38-ba4d-c64deb24c995 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:17.717822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77424c8-f362-4f38-ba4d-c64deb24c995 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:17.724784 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77424c8-f362-4f38-ba4d-c64deb24c995 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:17.726721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77424c8-f362-4f38-ba4d-c64deb24c995 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-a77424c8-f362-4f38-ba4d-c64deb24c995 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:08:17.738632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:17.742558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:17.743833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 605/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:17.746652 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87f5bd-5ed2-41ff-b706-ec2bdec82ef6 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:17.848780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87f5bd-5ed2-41ff-b706-ec2bdec82ef6 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:17.883675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8f316-bb25-447a-b5eb-61f9452253d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:17.884316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8f316-bb25-447a-b5eb-61f9452253d0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:17.884592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8f316-bb25-447a-b5eb-61f9452253d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:17.884860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8f316-bb25-447a-b5eb-61f9452253d0 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:17.885636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8f316-bb25-447a-b5eb-61f9452253d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:17.885940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8f316-bb25-447a-b5eb-61f9452253d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:17.887365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8f316-bb25-447a-b5eb-61f9452253d0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:17.903690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 608/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Aug 6 01:08:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:17.909571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77424c8-f362-4f38-ba4d-c64deb24c995 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:18.006855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77424c8-f362-4f38-ba4d-c64deb24c995 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:18.040030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913083d-a20f-4152-add0-c26cb52432f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:18.040647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913083d-a20f-4152-add0-c26cb52432f0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:18.040929 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913083d-a20f-4152-add0-c26cb52432f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:18.041204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913083d-a20f-4152-add0-c26cb52432f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:18.074107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:18.111659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913083d-a20f-4152-add0-c26cb52432f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:18.113130 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913083d-a20f-4152-add0-c26cb52432f0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:18.209362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 606/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:08:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:08:18.215678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8f316-bb25-447a-b5eb-61f9452253d0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:18.236889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:18.312597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8f316-bb25-447a-b5eb-61f9452253d0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:18.346768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9834d86-a5e8-4157-baea-ddc2a19dce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:18.347409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9834d86-a5e8-4157-baea-ddc2a19dce1c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:18.347692 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9834d86-a5e8-4157-baea-ddc2a19dce1c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:18.347994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9834d86-a5e8-4157-baea-ddc2a19dce1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:18.358037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9834d86-a5e8-4157-baea-ddc2a19dce1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:18.362811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9834d86-a5e8-4157-baea-ddc2a19dce1c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-f9834d86-a5e8-4157-baea-ddc2a19dce1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:18.373841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:18.374927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:18.376062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:18.376062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:18.376062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:18.376062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:18.376062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 609/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:18.381087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913083d-a20f-4152-add0-c26cb52432f0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:18.441132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:18.478950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913083d-a20f-4152-add0-c26cb52432f0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:18.511870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5670a19-012d-426d-8551-c3f17aed0463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:18.512484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5670a19-012d-426d-8551-c3f17aed0463 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:18.512769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5670a19-012d-426d-8551-c3f17aed0463 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:18.513049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5670a19-012d-426d-8551-c3f17aed0463 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:18.513794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5670a19-012d-426d-8551-c3f17aed0463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:18.514111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5670a19-012d-426d-8551-c3f17aed0463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:18.515499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5670a19-012d-426d-8551-c3f17aed0463 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:18.527794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 607/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:18.533532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9834d86-a5e8-4157-baea-ddc2a19dce1c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:18.601861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:18.631544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9834d86-a5e8-4157-baea-ddc2a19dce1c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:18.665391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb32e55-bc9c-46e8-82bc-5cc13fb705b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:18.666028 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb32e55-bc9c-46e8-82bc-5cc13fb705b5 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:18.666312 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb32e55-bc9c-46e8-82bc-5cc13fb705b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:18.666589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb32e55-bc9c-46e8-82bc-5cc13fb705b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:18.673029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb32e55-bc9c-46e8-82bc-5cc13fb705b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:18.674485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb32e55-bc9c-46e8-82bc-5cc13fb705b5 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-8cb32e55-bc9c-46e8-82bc-5cc13fb705b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:08:18.686500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:18.687696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:18.688643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:08:18.688643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:08:18.688643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:08:18.688643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:08:18.688643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:08:18.688643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:08:18.688643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:18.688643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:18.688643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 610/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:08:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:18.693577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5670a19-012d-426d-8551-c3f17aed0463 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:18.791476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5670a19-012d-426d-8551-c3f17aed0463 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:18.813680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:18.825375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840a25e-ff8a-4417-86c5-ed9a3d04d65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:18.825988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840a25e-ff8a-4417-86c5-ed9a3d04d65d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:18.826271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840a25e-ff8a-4417-86c5-ed9a3d04d65d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:18.826547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840a25e-ff8a-4417-86c5-ed9a3d04d65d None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:18.827295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840a25e-ff8a-4417-86c5-ed9a3d04d65d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:18.827607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840a25e-ff8a-4417-86c5-ed9a3d04d65d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:18.829086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840a25e-ff8a-4417-86c5-ed9a3d04d65d None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:18.840819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 608/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:08:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:18.847056 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb32e55-bc9c-46e8-82bc-5cc13fb705b5 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:18.945751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb32e55-bc9c-46e8-82bc-5cc13fb705b5 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:18.972968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:18.978617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370753c-4bfb-44df-ac15-3f0a20a3ce5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:18.979232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370753c-4bfb-44df-ac15-3f0a20a3ce5e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:18.979515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370753c-4bfb-44df-ac15-3f0a20a3ce5e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:18.979803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370753c-4bfb-44df-ac15-3f0a20a3ce5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:18.980605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370753c-4bfb-44df-ac15-3f0a20a3ce5e None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=8361f2f9c70b4a3288a187cb2d97d415) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:18.980921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370753c-4bfb-44df-ac15-3f0a20a3ce5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:18.982373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370753c-4bfb-44df-ac15-3f0a20a3ce5e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:18.987572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 611/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:08:18 2022] GET /identity/v3/role_assignments?user.id=8361f2f9c70b4a3288a187cb2d97d415&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:20.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9315ce-a59a-45c7-8356-135e6314595c None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:20.982266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9315ce-a59a-45c7-8356-135e6314595c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:20.982379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9315ce-a59a-45c7-8356-135e6314595c None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:20.983648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 609/1220] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:08:20.988840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1667c4-08fe-46c2-a831-caeb96af1ec9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:20.989275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1667c4-08fe-46c2-a831-caeb96af1ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:20.989394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1667c4-08fe-46c2-a831-caeb96af1ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:21.009540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6c1667c4-08fe-46c2-a831-caeb96af1ec9 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:21.011216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6c1667c4-08fe-46c2-a831-caeb96af1ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:21.082196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 612/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:08:21.104861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9315ce-a59a-45c7-8356-135e6314595c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:21.214244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9315ce-a59a-45c7-8356-135e6314595c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:21.249970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe46248-cef3-4785-b355-62195f0bb81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:21.250531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe46248-cef3-4785-b355-62195f0bb81d None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:21.250810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe46248-cef3-4785-b355-62195f0bb81d None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:21.251082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe46248-cef3-4785-b355-62195f0bb81d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:21.264455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe46248-cef3-4785-b355-62195f0bb81d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-bbe46248-cef3-4785-b355-62195f0bb81d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:08:21.269361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:21.271209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:21.272412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 610/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:08:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:21.276640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1667c4-08fe-46c2-a831-caeb96af1ec9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:21.375465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1667c4-08fe-46c2-a831-caeb96af1ec9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:21.397591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:21.407509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf7c8f-ad52-44d4-af35-d17c0ee01a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:21.408166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf7c8f-ad52-44d4-af35-d17c0ee01a9d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:21.408449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf7c8f-ad52-44d4-af35-d17c0ee01a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:21.408720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf7c8f-ad52-44d4-af35-d17c0ee01a9d None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:21.409434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf7c8f-ad52-44d4-af35-d17c0ee01a9d None admin] RBAC: Adding query filter params (name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:21.409738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf7c8f-ad52-44d4-af35-d17c0ee01a9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:21.411099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf7c8f-ad52-44d4-af35-d17c0ee01a9d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:21.419096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 613/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:08:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:21.425119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe46248-cef3-4785-b355-62195f0bb81d None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:21.473245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:21.527269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe46248-cef3-4785-b355-62195f0bb81d None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:21.559967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c08843-97f2-4484-9416-788e3832e7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:21.560597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c08843-97f2-4484-9416-788e3832e7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:21.560880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c08843-97f2-4484-9416-788e3832e7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:21.561158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c08843-97f2-4484-9416-788e3832e7c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:21.571424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c08843-97f2-4484-9416-788e3832e7c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:21.572877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c08843-97f2-4484-9416-788e3832e7c9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-84c08843-97f2-4484-9416-788e3832e7c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:21.586996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:21.587978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:21.588888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:21.588888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:21.588888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:21.588888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:21.591950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 611/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:21.594250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf7c8f-ad52-44d4-af35-d17c0ee01a9d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:21.691984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf7c8f-ad52-44d4-af35-d17c0ee01a9d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:21.724061 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2446970-b6ef-42a5-8a77-9e5b990311ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:21.724676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2446970-b6ef-42a5-8a77-9e5b990311ce None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:21.724953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2446970-b6ef-42a5-8a77-9e5b990311ce None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:21.725222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2446970-b6ef-42a5-8a77-9e5b990311ce None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:21.726489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2446970-b6ef-42a5-8a77-9e5b990311ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:21.726792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2446970-b6ef-42a5-8a77-9e5b990311ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:21.728170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2446970-b6ef-42a5-8a77-9e5b990311ce None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:21.739622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 614/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:21.745273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c08843-97f2-4484-9416-788e3832e7c9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:21.837431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:21.843941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c08843-97f2-4484-9416-788e3832e7c9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:21.876834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72104c-62cd-4059-8a70-c953b6548ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:21.877852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72104c-62cd-4059-8a70-c953b6548ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:21.878151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72104c-62cd-4059-8a70-c953b6548ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:21.878800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72104c-62cd-4059-8a70-c953b6548ff7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:21.887034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72104c-62cd-4059-8a70-c953b6548ff7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:21.888551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72104c-62cd-4059-8a70-c953b6548ff7 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-5b72104c-62cd-4059-8a70-c953b6548ff7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:08:21.899801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:21.900830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:21.901832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 612/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:21.906928 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2446970-b6ef-42a5-8a77-9e5b990311ce None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:21.952488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:22.003066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2446970-b6ef-42a5-8a77-9e5b990311ce None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:22.037832 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00724c-7304-4a31-9f12-3e321b601b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:22.038932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00724c-7304-4a31-9f12-3e321b601b90 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:22.039213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00724c-7304-4a31-9f12-3e321b601b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:22.039488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00724c-7304-4a31-9f12-3e321b601b90 None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:22.040346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00724c-7304-4a31-9f12-3e321b601b90 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:22.048162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00724c-7304-4a31-9f12-3e321b601b90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:22.048162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00724c-7304-4a31-9f12-3e321b601b90 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:22.059008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 615/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Aug 6 01:08:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:22.064786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72104c-62cd-4059-8a70-c953b6548ff7 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:22.163028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72104c-62cd-4059-8a70-c953b6548ff7 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:22.195560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e6371-b5f1-4b14-bf44-27391b2498df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:22.196217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6371-b5f1-4b14-bf44-27391b2498df None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:22.196501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6371-b5f1-4b14-bf44-27391b2498df None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:22.196776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6371-b5f1-4b14-bf44-27391b2498df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:22.207831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e6371-b5f1-4b14-bf44-27391b2498df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:22.209539 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e6371-b5f1-4b14-bf44-27391b2498df None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-845e6371-b5f1-4b14-bf44-27391b2498df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:22.224036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:22.225153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:22.226324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:22.226324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:22.226324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:22.226324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:22.226324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 613/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:22.230906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00724c-7304-4a31-9f12-3e321b601b90 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:22.326924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00724c-7304-4a31-9f12-3e321b601b90 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:22.329401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:22.359944 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4402401-4ee1-4cc9-86b1-de72580a78dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:22.360561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4402401-4ee1-4cc9-86b1-de72580a78dc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:22.360843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4402401-4ee1-4cc9-86b1-de72580a78dc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:22.361118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4402401-4ee1-4cc9-86b1-de72580a78dc None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:22.361795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4402401-4ee1-4cc9-86b1-de72580a78dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:22.362098 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4402401-4ee1-4cc9-86b1-de72580a78dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:22.363459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4402401-4ee1-4cc9-86b1-de72580a78dc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:22.374017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 616/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:22.379665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e6371-b5f1-4b14-bf44-27391b2498df None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:22.389230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:22.499439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e6371-b5f1-4b14-bf44-27391b2498df None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:22.536990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a144e-4b1c-4125-b564-7d2082648434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:22.537626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a144e-4b1c-4125-b564-7d2082648434 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:22.537925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a144e-4b1c-4125-b564-7d2082648434 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:22.538203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a144e-4b1c-4125-b564-7d2082648434 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:22.544798 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a144e-4b1c-4125-b564-7d2082648434 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:22.546234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a144e-4b1c-4125-b564-7d2082648434 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-208a144e-4b1c-4125-b564-7d2082648434 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:08:22.557909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:22.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:22.560160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:08:22.560160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:08:22.560160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:08:22.560160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:08:22.560160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:08:22.560160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:08:22.560160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:22.560160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:22.560160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 614/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:08:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:22.565133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4402401-4ee1-4cc9-86b1-de72580a78dc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:22.664700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4402401-4ee1-4cc9-86b1-de72580a78dc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:22.698260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63284a7e-fbdd-4860-aacc-35f884a99476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:22.698882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63284a7e-fbdd-4860-aacc-35f884a99476 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:22.699166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63284a7e-fbdd-4860-aacc-35f884a99476 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:22.699443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63284a7e-fbdd-4860-aacc-35f884a99476 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:22.700241 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63284a7e-fbdd-4860-aacc-35f884a99476 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:22.700551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63284a7e-fbdd-4860-aacc-35f884a99476 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:22.702049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63284a7e-fbdd-4860-aacc-35f884a99476 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:22.714188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 617/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:08:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:22.720245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a144e-4b1c-4125-b564-7d2082648434 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:22.785019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:22.822956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a144e-4b1c-4125-b564-7d2082648434 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:22.855724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8bd4e0-250f-4554-bf88-5702235c0683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:22.856407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bd4e0-250f-4554-bf88-5702235c0683 None admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:22.856688 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bd4e0-250f-4554-bf88-5702235c0683 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:22.856964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8bd4e0-250f-4554-bf88-5702235c0683 None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/8361f2f9c70b4a3288a187cb2d97d415/roles/4dabc11cde5340ee9eb2c28e6154b3ec` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:22.871202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8bd4e0-250f-4554-bf88-5702235c0683 None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=8361f2f9c70b4a3288a187cb2d97d415, role_id=4dabc11cde5340ee9eb2c28e6154b3ec)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:22.872663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8bd4e0-250f-4554-bf88-5702235c0683 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:22.969154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 615/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:08:22 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/8361f2f9c70b4a3288a187cb2d97d415/roles/4dabc11cde5340ee9eb2c28e6154b3ec => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:08:24.916830 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf4b68-557b-45e2-9338-217a12286e00 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:24.917340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf4b68-557b-45e2-9338-217a12286e00 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:24.917340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf4b68-557b-45e2-9338-217a12286e00 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:24.918676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 618/1233] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:08:24.923651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77d0c-12da-4fa9-a5ed-e767f0c41195 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:24.924047 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77d0c-12da-4fa9-a5ed-e767f0c41195 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:24.924248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77d0c-12da-4fa9-a5ed-e767f0c41195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:24.941585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-45f77d0c-12da-4fa9-a5ed-e767f0c41195 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:24.943293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-45f77d0c-12da-4fa9-a5ed-e767f0c41195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:25.006728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 616/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:08:25.029897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf4b68-557b-45e2-9338-217a12286e00 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:25.055321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:25.128877 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf4b68-557b-45e2-9338-217a12286e00 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:25.163522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4956db-c527-44eb-8cc5-6c74a5cd614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:25.164272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4956db-c527-44eb-8cc5-6c74a5cd614d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:25.164559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4956db-c527-44eb-8cc5-6c74a5cd614d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:25.164837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4956db-c527-44eb-8cc5-6c74a5cd614d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:25.175350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4956db-c527-44eb-8cc5-6c74a5cd614d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-ad4956db-c527-44eb-8cc5-6c74a5cd614d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:08:25.180367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:25.185242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:25.186301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 619/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Sat Aug 6 01:08:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:25.187636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f77d0c-12da-4fa9-a5ed-e767f0c41195 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:25.234440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:25.285312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f77d0c-12da-4fa9-a5ed-e767f0c41195 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:25.318180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75990f-d5e5-4dea-9b0b-8a263af3f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:25.318786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75990f-d5e5-4dea-9b0b-8a263af3f363 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:25.319561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75990f-d5e5-4dea-9b0b-8a263af3f363 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:25.319863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75990f-d5e5-4dea-9b0b-8a263af3f363 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:25.320575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75990f-d5e5-4dea-9b0b-8a263af3f363 None admin] RBAC: Adding query filter params (name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:25.320881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75990f-d5e5-4dea-9b0b-8a263af3f363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:25.324005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75990f-d5e5-4dea-9b0b-8a263af3f363 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:25.331990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 617/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Sat Aug 6 01:08:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:25.339012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4956db-c527-44eb-8cc5-6c74a5cd614d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:25.419702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:25.435442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4956db-c527-44eb-8cc5-6c74a5cd614d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:25.468105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f6b73-d969-49e8-a9e2-397d5f5545c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:25.468997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f6b73-d969-49e8-a9e2-397d5f5545c4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:25.469730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f6b73-d969-49e8-a9e2-397d5f5545c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:25.470005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f6b73-d969-49e8-a9e2-397d5f5545c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:25.479708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f6b73-d969-49e8-a9e2-397d5f5545c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:25.481427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f6b73-d969-49e8-a9e2-397d5f5545c4 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-0b9f6b73-d969-49e8-a9e2-397d5f5545c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:25.497017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:25.504519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:25.505619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:25.505619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:25.505619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:25.505619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:25.505619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 620/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:08:25.505974 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75990f-d5e5-4dea-9b0b-8a263af3f363 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:25.603449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75990f-d5e5-4dea-9b0b-8a263af3f363 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:25.612229 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:25.636780 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcbd6-240e-40bb-a647-521f92ae0f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:25.637397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcbd6-240e-40bb-a647-521f92ae0f49 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:25.637674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcbd6-240e-40bb-a647-521f92ae0f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:25.637952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcbd6-240e-40bb-a647-521f92ae0f49 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:25.638639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bcbd6-240e-40bb-a647-521f92ae0f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:25.638941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bcbd6-240e-40bb-a647-521f92ae0f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:25.640331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00bcbd6-240e-40bb-a647-521f92ae0f49 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:25.651594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 618/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:25.657415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f6b73-d969-49e8-a9e2-397d5f5545c4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:25.753474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f6b73-d969-49e8-a9e2-397d5f5545c4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:25.786271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024436a9-faf7-4196-a176-eae017ebb9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:25.786899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024436a9-faf7-4196-a176-eae017ebb9ad None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:25.787177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024436a9-faf7-4196-a176-eae017ebb9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:25.787449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024436a9-faf7-4196-a176-eae017ebb9ad None admin] PATH_INFO: `/v3/users/placement` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:25.790614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:25.795480 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024436a9-faf7-4196-a176-eae017ebb9ad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:25.797043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024436a9-faf7-4196-a176-eae017ebb9ad None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-024436a9-faf7-4196-a176-eae017ebb9ad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:08:25.808596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:25.809691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:25.815398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 621/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:25.816184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcbd6-240e-40bb-a647-521f92ae0f49 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:25.911541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcbd6-240e-40bb-a647-521f92ae0f49 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:25.945142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1ff0f-86bc-4949-a163-9c639ae792e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:25.945736 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1ff0f-86bc-4949-a163-9c639ae792e0 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:25.946013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1ff0f-86bc-4949-a163-9c639ae792e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:25.946344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c1ff0f-86bc-4949-a163-9c639ae792e0 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:25.947065 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1ff0f-86bc-4949-a163-9c639ae792e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:25.947369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1ff0f-86bc-4949-a163-9c639ae792e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:25.948805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c1ff0f-86bc-4949-a163-9c639ae792e0 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:25.965259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 619/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Aug 6 01:08:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:25.970982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024436a9-faf7-4196-a176-eae017ebb9ad None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:26.070432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024436a9-faf7-4196-a176-eae017ebb9ad None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:26.103819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220ce3f-c3e9-40cc-a1c1-e13b1d2107f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:26.104648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220ce3f-c3e9-40cc-a1c1-e13b1d2107f0 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:26.104932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220ce3f-c3e9-40cc-a1c1-e13b1d2107f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:26.105212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220ce3f-c3e9-40cc-a1c1-e13b1d2107f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:26.166494 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:26.172901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220ce3f-c3e9-40cc-a1c1-e13b1d2107f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:26.174366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220ce3f-c3e9-40cc-a1c1-e13b1d2107f0 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:26.267147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 622/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:08:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:08:26.273355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1ff0f-86bc-4949-a163-9c639ae792e0 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:26.300089 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:26.370108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c1ff0f-86bc-4949-a163-9c639ae792e0 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:26.402977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8163f53-926f-48f5-9a47-3941a497dafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:26.403606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163f53-926f-48f5-9a47-3941a497dafe None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:26.403918 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163f53-926f-48f5-9a47-3941a497dafe None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:26.404198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163f53-926f-48f5-9a47-3941a497dafe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:26.414528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8163f53-926f-48f5-9a47-3941a497dafe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:26.415975 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8163f53-926f-48f5-9a47-3941a497dafe None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-d8163f53-926f-48f5-9a47-3941a497dafe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:26.430125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:26.432252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:26.433357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:26.433357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:26.433357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:26.433357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:26.433357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 620/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:26.437104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220ce3f-c3e9-40cc-a1c1-e13b1d2107f0 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:26.528169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:26.533855 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220ce3f-c3e9-40cc-a1c1-e13b1d2107f0 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:26.566882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec85fc-3e56-4671-bdee-1a56d4191322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:26.567582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec85fc-3e56-4671-bdee-1a56d4191322 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:26.567901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec85fc-3e56-4671-bdee-1a56d4191322 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:26.568184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec85fc-3e56-4671-bdee-1a56d4191322 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:26.568871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec85fc-3e56-4671-bdee-1a56d4191322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:26.569251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec85fc-3e56-4671-bdee-1a56d4191322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:26.570629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec85fc-3e56-4671-bdee-1a56d4191322 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:26.581511 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 623/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:26.587421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8163f53-926f-48f5-9a47-3941a497dafe None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:26.673757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:26.694749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8163f53-926f-48f5-9a47-3941a497dafe None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:26.727137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafed83a-805f-4f3f-ac80-d85b7e96c2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:26.727765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafed83a-805f-4f3f-ac80-d85b7e96c2ce None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:26.728101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafed83a-805f-4f3f-ac80-d85b7e96c2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:26.728378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafed83a-805f-4f3f-ac80-d85b7e96c2ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:26.735220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafed83a-805f-4f3f-ac80-d85b7e96c2ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:26.736694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafed83a-805f-4f3f-ac80-d85b7e96c2ce None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-dafed83a-805f-4f3f-ac80-d85b7e96c2ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:08:26.748461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:26.749348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:26.750518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:08:26.750518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:08:26.750518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:08:26.750518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:08:26.750518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:08:26.750518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:08:26.750518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:26.750518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:26.750518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 621/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:08:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:26.755437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec85fc-3e56-4671-bdee-1a56d4191322 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:26.853293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec85fc-3e56-4671-bdee-1a56d4191322 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:26.885909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b23cd-e46f-44ef-89dc-578dec2d57cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:26.886536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b23cd-e46f-44ef-89dc-578dec2d57cb None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:26.886820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b23cd-e46f-44ef-89dc-578dec2d57cb None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:26.887124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b23cd-e46f-44ef-89dc-578dec2d57cb None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:26.887894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b23cd-e46f-44ef-89dc-578dec2d57cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:26.888203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b23cd-e46f-44ef-89dc-578dec2d57cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:26.889640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b23cd-e46f-44ef-89dc-578dec2d57cb None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:26.901275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 624/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:08:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:26.907343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafed83a-805f-4f3f-ac80-d85b7e96c2ce None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:27.010313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafed83a-805f-4f3f-ac80-d85b7e96c2ce None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:27.043519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11ccb9-7dab-4ff8-a6d7-e6ba36e06087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:27.044158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11ccb9-7dab-4ff8-a6d7-e6ba36e06087 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:27.044438 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11ccb9-7dab-4ff8-a6d7-e6ba36e06087 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:27.044712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11ccb9-7dab-4ff8-a6d7-e6ba36e06087 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:27.045497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11ccb9-7dab-4ff8-a6d7-e6ba36e06087 None admin] RBAC: Adding query filter params (role.id=4dabc11cde5340ee9eb2c28e6154b3ec, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=8361f2f9c70b4a3288a187cb2d97d415) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:27.045808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11ccb9-7dab-4ff8-a6d7-e6ba36e06087 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:27.047209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11ccb9-7dab-4ff8-a6d7-e6ba36e06087 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:27.052900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 622/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:08:26 2022] GET /identity/v3/role_assignments?user.id=8361f2f9c70b4a3288a187cb2d97d415&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=4dabc11cde5340ee9eb2c28e6154b3ec => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:29.085513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a3460-3e32-4725-940e-ca1b8acb4524 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:29.086120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a3460-3e32-4725-940e-ca1b8acb4524 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:29.086120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a3460-3e32-4725-940e-ca1b8acb4524 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:29.087446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 625/1247] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:08:29.092655 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc6f0-6a90-4b1b-bd32-b7db4bb8e741 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:29.093098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc6f0-6a90-4b1b-bd32-b7db4bb8e741 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:29.093241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc6f0-6a90-4b1b-bd32-b7db4bb8e741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:29.099449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:29.111227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ffebc6f0-6a90-4b1b-bd32-b7db4bb8e741 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:29.112441 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ffebc6f0-6a90-4b1b-bd32-b7db4bb8e741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:29.175241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 623/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 01:08:29.199273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a3460-3e32-4725-940e-ca1b8acb4524 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:29.217526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:29.297168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a3460-3e32-4725-940e-ca1b8acb4524 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:29.330951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d1a97-4f24-48b9-981b-16a85552cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:29.331590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d1a97-4f24-48b9-981b-16a85552cf38 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:29.331902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d1a97-4f24-48b9-981b-16a85552cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:29.332181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d1a97-4f24-48b9-981b-16a85552cf38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:29.346084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d1a97-4f24-48b9-981b-16a85552cf38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-593d1a97-4f24-48b9-981b-16a85552cf38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:08:29.351075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:29.352238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:29.358164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 626/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:08:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:29.359018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebc6f0-6a90-4b1b-bd32-b7db4bb8e741 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:29.454541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebc6f0-6a90-4b1b-bd32-b7db4bb8e741 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:29.482486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:29.487197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46831c4-a2f2-4be9-b8a9-1072914f46f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:29.487826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46831c4-a2f2-4be9-b8a9-1072914f46f6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:29.488111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46831c4-a2f2-4be9-b8a9-1072914f46f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:29.488386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46831c4-a2f2-4be9-b8a9-1072914f46f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:29.489105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46831c4-a2f2-4be9-b8a9-1072914f46f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:29.489407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46831c4-a2f2-4be9-b8a9-1072914f46f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:29.491273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46831c4-a2f2-4be9-b8a9-1072914f46f6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:29.498993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 624/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:08:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:08:29.505032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d1a97-4f24-48b9-981b-16a85552cf38 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:29.581729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:29.604509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d1a97-4f24-48b9-981b-16a85552cf38 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:29.636828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372d8c5-f976-432f-bfc9-84e56755155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:29.637465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372d8c5-f976-432f-bfc9-84e56755155e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:29.637748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372d8c5-f976-432f-bfc9-84e56755155e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:29.638021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372d8c5-f976-432f-bfc9-84e56755155e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:29.648589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372d8c5-f976-432f-bfc9-84e56755155e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:29.650288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372d8c5-f976-432f-bfc9-84e56755155e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-d372d8c5-f976-432f-bfc9-84e56755155e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:29.664513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:29.665578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:29.666569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:29.666569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:29.666569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:29.666569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:29.666569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 627/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:29.671547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46831c4-a2f2-4be9-b8a9-1072914f46f6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:29.768779 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46831c4-a2f2-4be9-b8a9-1072914f46f6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:29.801962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735e10b-b5a2-4975-99f8-19673a3f9f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:29.802568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735e10b-b5a2-4975-99f8-19673a3f9f3a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:29.802847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735e10b-b5a2-4975-99f8-19673a3f9f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:29.803119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735e10b-b5a2-4975-99f8-19673a3f9f3a None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:29.803889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735e10b-b5a2-4975-99f8-19673a3f9f3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:29.804192 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735e10b-b5a2-4975-99f8-19673a3f9f3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:29.805538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735e10b-b5a2-4975-99f8-19673a3f9f3a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:29.816922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 625/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:29.823205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372d8c5-f976-432f-bfc9-84e56755155e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:29.925388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372d8c5-f976-432f-bfc9-84e56755155e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:29.951545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:29.959542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c90666-6eac-4607-baa8-583d14e44d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:29.960212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c90666-6eac-4607-baa8-583d14e44d6c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:29.960490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c90666-6eac-4607-baa8-583d14e44d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:29.960758 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c90666-6eac-4607-baa8-583d14e44d6c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:29.968087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c90666-6eac-4607-baa8-583d14e44d6c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:29.969515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c90666-6eac-4607-baa8-583d14e44d6c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-34c90666-6eac-4607-baa8-583d14e44d6c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:08:29.980611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:29.981733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:29.982901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 628/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:29.987618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735e10b-b5a2-4975-99f8-19673a3f9f3a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:30.042338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:30.084557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735e10b-b5a2-4975-99f8-19673a3f9f3a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:30.118189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f8f7d-ac18-4fc5-a76c-8708962488b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:30.118810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f8f7d-ac18-4fc5-a76c-8708962488b9 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:30.119067 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f8f7d-ac18-4fc5-a76c-8708962488b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:30.119338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f8f7d-ac18-4fc5-a76c-8708962488b9 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:30.120159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f8f7d-ac18-4fc5-a76c-8708962488b9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:30.120469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f8f7d-ac18-4fc5-a76c-8708962488b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:30.121883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f8f7d-ac18-4fc5-a76c-8708962488b9 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:30.139014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 626/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Aug 6 01:08:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:30.145003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c90666-6eac-4607-baa8-583d14e44d6c None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:30.240451 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c90666-6eac-4607-baa8-583d14e44d6c None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:30.277439 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59ab62-0303-4fa8-a987-89c477ac3d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:30.278027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ab62-0303-4fa8-a987-89c477ac3d34 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:30.278337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ab62-0303-4fa8-a987-89c477ac3d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:30.278611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ab62-0303-4fa8-a987-89c477ac3d34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:30.332661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:30.344761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59ab62-0303-4fa8-a987-89c477ac3d34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:30.346229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59ab62-0303-4fa8-a987-89c477ac3d34 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:30.442524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 629/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:08:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:08:30.448351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f8f7d-ac18-4fc5-a76c-8708962488b9 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:30.544378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f8f7d-ac18-4fc5-a76c-8708962488b9 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:30.558725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:30.579626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307ee45-011b-4bc9-9990-312c29698cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:30.580299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307ee45-011b-4bc9-9990-312c29698cca None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:30.580580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307ee45-011b-4bc9-9990-312c29698cca None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:30.580858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307ee45-011b-4bc9-9990-312c29698cca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:30.591032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307ee45-011b-4bc9-9990-312c29698cca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:30.592511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307ee45-011b-4bc9-9990-312c29698cca None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-6307ee45-011b-4bc9-9990-312c29698cca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:30.607227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:30.608324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:30.609447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:30.609447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:30.609447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:30.609447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:30.609447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 627/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:30.614187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59ab62-0303-4fa8-a987-89c477ac3d34 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:30.698834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:30.711263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59ab62-0303-4fa8-a987-89c477ac3d34 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:30.744171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e20da-94a4-4ea6-957a-d0cb49ddba93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:30.744792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e20da-94a4-4ea6-957a-d0cb49ddba93 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:30.745074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e20da-94a4-4ea6-957a-d0cb49ddba93 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:30.745348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e20da-94a4-4ea6-957a-d0cb49ddba93 None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:30.746035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e20da-94a4-4ea6-957a-d0cb49ddba93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:30.746343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e20da-94a4-4ea6-957a-d0cb49ddba93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:30.747731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e20da-94a4-4ea6-957a-d0cb49ddba93 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:30.758380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 630/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:30.764087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307ee45-011b-4bc9-9990-312c29698cca None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:30.863772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307ee45-011b-4bc9-9990-312c29698cca None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:30.895482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f13b7-49f4-4e53-9f60-a68163adcbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:30.896137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f13b7-49f4-4e53-9f60-a68163adcbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:30.896422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f13b7-49f4-4e53-9f60-a68163adcbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:30.896698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f13b7-49f4-4e53-9f60-a68163adcbb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:30.903950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f13b7-49f4-4e53-9f60-a68163adcbb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:30.905389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f13b7-49f4-4e53-9f60-a68163adcbb6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-293f13b7-49f4-4e53-9f60-a68163adcbb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:08:30.917146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:30.918178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:30.919117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:08:30.919117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:08:30.919117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:08:30.919117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:08:30.919117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:08:30.919117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:08:30.919117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:30.919117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:30.919117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 628/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:08:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:30.924169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e20da-94a4-4ea6-957a-d0cb49ddba93 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:31.019991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e20da-94a4-4ea6-957a-d0cb49ddba93 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:31.052744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8d1d-601c-4e26-b448-09347f4be635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:31.053390 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8d1d-601c-4e26-b448-09347f4be635 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:31.053675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8d1d-601c-4e26-b448-09347f4be635 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:31.053953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8d1d-601c-4e26-b448-09347f4be635 None admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:31.054699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8d1d-601c-4e26-b448-09347f4be635 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:31.055011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8d1d-601c-4e26-b448-09347f4be635 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:31.056470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8d1d-601c-4e26-b448-09347f4be635 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:31.067910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 631/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:08:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:08:31.074462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f13b7-49f4-4e53-9f60-a68163adcbb6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:31.101411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:31.173374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f13b7-49f4-4e53-9f60-a68163adcbb6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:31.208285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e41ce-8f57-4259-8da4-ff75f7b9c444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:31.208894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e41ce-8f57-4259-8da4-ff75f7b9c444 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:31.209180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e41ce-8f57-4259-8da4-ff75f7b9c444 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:31.209465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e41ce-8f57-4259-8da4-ff75f7b9c444 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:31.210794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e41ce-8f57-4259-8da4-ff75f7b9c444 None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=8361f2f9c70b4a3288a187cb2d97d415) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:31.211100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e41ce-8f57-4259-8da4-ff75f7b9c444 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:31.212528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e41ce-8f57-4259-8da4-ff75f7b9c444 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:31.217782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 629/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:08:31 2022] GET /identity/v3/role_assignments?user.id=8361f2f9c70b4a3288a187cb2d97d415&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:33.178898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04dad7-9250-4221-92d1-83de1d591417 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:33.179447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04dad7-9250-4221-92d1-83de1d591417 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:33.179447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04dad7-9250-4221-92d1-83de1d591417 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:33.180832 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 632/1261] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:08:33.186059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e8b9e-1581-4508-be86-8dcf3c4ffd0c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:33.186455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e8b9e-1581-4508-be86-8dcf3c4ffd0c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:33.186605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e8b9e-1581-4508-be86-8dcf3c4ffd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:33.206460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-234e8b9e-1581-4508-be86-8dcf3c4ffd0c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:33.208264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-234e8b9e-1581-4508-be86-8dcf3c4ffd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:33.273558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 630/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:08:33.296123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04dad7-9250-4221-92d1-83de1d591417 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:33.307428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:33.396415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04dad7-9250-4221-92d1-83de1d591417 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:33.433506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03fb69-625f-41c2-bde9-ec3ec8b9f7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:33.434071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03fb69-625f-41c2-bde9-ec3ec8b9f7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:33.434350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03fb69-625f-41c2-bde9-ec3ec8b9f7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:33.434622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03fb69-625f-41c2-bde9-ec3ec8b9f7a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:33.444651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03fb69-625f-41c2-bde9-ec3ec8b9f7a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-af03fb69-625f-41c2-bde9-ec3ec8b9f7a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:08:33.449436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:33.450569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:33.451691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 633/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:08:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:33.456706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e8b9e-1581-4508-be86-8dcf3c4ffd0c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:33.475689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:33.552859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e8b9e-1581-4508-be86-8dcf3c4ffd0c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:33.589013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b780-379b-4022-a31e-adf9cb5f4bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:33.589638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b780-379b-4022-a31e-adf9cb5f4bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:33.589910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b780-379b-4022-a31e-adf9cb5f4bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:33.590186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b780-379b-4022-a31e-adf9cb5f4bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:33.590904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53b780-379b-4022-a31e-adf9cb5f4bf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:33.591213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53b780-379b-4022-a31e-adf9cb5f4bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:33.592607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53b780-379b-4022-a31e-adf9cb5f4bf5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:33.601423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 631/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:08:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:33.607411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03fb69-625f-41c2-bde9-ec3ec8b9f7a4 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:33.673482 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:33.706732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03fb69-625f-41c2-bde9-ec3ec8b9f7a4 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:33.740959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c26420-2115-4d1b-82cd-2b56a8ce473e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:33.741591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c26420-2115-4d1b-82cd-2b56a8ce473e None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:33.741868 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c26420-2115-4d1b-82cd-2b56a8ce473e None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:33.742138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c26420-2115-4d1b-82cd-2b56a8ce473e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:33.751622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c26420-2115-4d1b-82cd-2b56a8ce473e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:33.753097 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c26420-2115-4d1b-82cd-2b56a8ce473e None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-72c26420-2115-4d1b-82cd-2b56a8ce473e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:33.766988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:33.768031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:33.769128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:33.769128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:33.769128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:33.769128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:33.769128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 634/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:33.774374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b780-379b-4022-a31e-adf9cb5f4bf5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:33.855157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:33.873404 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b780-379b-4022-a31e-adf9cb5f4bf5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:33.906322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15892c1-a848-4d16-8944-7ce3681cef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:33.906933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15892c1-a848-4d16-8944-7ce3681cef1f None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:33.907210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15892c1-a848-4d16-8944-7ce3681cef1f None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:33.907480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15892c1-a848-4d16-8944-7ce3681cef1f None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:33.908192 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15892c1-a848-4d16-8944-7ce3681cef1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:33.908494 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15892c1-a848-4d16-8944-7ce3681cef1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:33.909847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15892c1-a848-4d16-8944-7ce3681cef1f None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:33.920729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 632/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:33.926603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c26420-2115-4d1b-82cd-2b56a8ce473e None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:34.025242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c26420-2115-4d1b-82cd-2b56a8ce473e None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:34.046082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:34.060207 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7564a-1da9-4d82-985b-691ac2abd134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:34.060920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7564a-1da9-4d82-985b-691ac2abd134 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:34.061199 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7564a-1da9-4d82-985b-691ac2abd134 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:34.061468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7564a-1da9-4d82-985b-691ac2abd134 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:34.069572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7564a-1da9-4d82-985b-691ac2abd134 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:34.071005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7564a-1da9-4d82-985b-691ac2abd134 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-3da7564a-1da9-4d82-985b-691ac2abd134 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:08:34.082075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:34.083285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:34.084549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 635/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:34.089387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15892c1-a848-4d16-8944-7ce3681cef1f None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:34.188053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15892c1-a848-4d16-8944-7ce3681cef1f None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:34.220804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdab4a7-a158-4eb3-a94b-a0856f2c30b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:34.221413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdab4a7-a158-4eb3-a94b-a0856f2c30b6 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:34.221691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdab4a7-a158-4eb3-a94b-a0856f2c30b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:34.221963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdab4a7-a158-4eb3-a94b-a0856f2c30b6 None admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:34.222743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdab4a7-a158-4eb3-a94b-a0856f2c30b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:34.223047 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdab4a7-a158-4eb3-a94b-a0856f2c30b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:34.224550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdab4a7-a158-4eb3-a94b-a0856f2c30b6 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:34.234158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:34.242599 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 633/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Aug 6 01:08:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:34.248316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7564a-1da9-4d82-985b-691ac2abd134 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:34.344716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7564a-1da9-4d82-985b-691ac2abd134 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:34.377502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7f174-51db-4d0d-8d40-7a394067b12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:34.378125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7f174-51db-4d0d-8d40-7a394067b12d None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:34.378403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7f174-51db-4d0d-8d40-7a394067b12d None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:34.378670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7f174-51db-4d0d-8d40-7a394067b12d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:34.388671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7f174-51db-4d0d-8d40-7a394067b12d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:34.390105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7f174-51db-4d0d-8d40-7a394067b12d None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-e9b7f174-51db-4d0d-8d40-7a394067b12d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:34.404170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:34.408772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:34.409936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:34.409936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:34.409936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:34.409936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:34.409936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 636/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:34.411140 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdab4a7-a158-4eb3-a94b-a0856f2c30b6 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:34.507609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdab4a7-a158-4eb3-a94b-a0856f2c30b6 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:34.541045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b994b-3256-430b-adf2-4db69b6ce0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:34.541651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b994b-3256-430b-adf2-4db69b6ce0fa None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:34.541932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b994b-3256-430b-adf2-4db69b6ce0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:34.542207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b994b-3256-430b-adf2-4db69b6ce0fa None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:34.542895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b994b-3256-430b-adf2-4db69b6ce0fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:34.543201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b994b-3256-430b-adf2-4db69b6ce0fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:34.544594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b994b-3256-430b-adf2-4db69b6ce0fa None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:34.555385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 634/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:08:34.561058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7f174-51db-4d0d-8d40-7a394067b12d None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:34.593281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:34.661314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7f174-51db-4d0d-8d40-7a394067b12d None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:34.694475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02eacb7-3775-438a-840f-77a91632b7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:34.695114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02eacb7-3775-438a-840f-77a91632b7ba None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:34.695400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02eacb7-3775-438a-840f-77a91632b7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:34.695669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02eacb7-3775-438a-840f-77a91632b7ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:34.702279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02eacb7-3775-438a-840f-77a91632b7ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:34.703795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02eacb7-3775-438a-840f-77a91632b7ba None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-b02eacb7-3775-438a-840f-77a91632b7ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:08:34.715598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:34.716572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:34.717626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:08:34.717626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:08:34.717626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:08:34.717626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:08:34.717626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:08:34.717626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:08:34.717626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:34.717626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:34.717626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 637/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:34.722664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b994b-3256-430b-adf2-4db69b6ce0fa None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:34.780514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:34.823338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b994b-3256-430b-adf2-4db69b6ce0fa None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:34.857610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6f8bd-a314-49be-84bb-193100c95ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:34.858224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6f8bd-a314-49be-84bb-193100c95ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:34.858511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6f8bd-a314-49be-84bb-193100c95ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:34.858794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6f8bd-a314-49be-84bb-193100c95ef2 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:34.859541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6f8bd-a314-49be-84bb-193100c95ef2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:34.859890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6f8bd-a314-49be-84bb-193100c95ef2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:34.861382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6f8bd-a314-49be-84bb-193100c95ef2 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:34.873077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 635/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:08:34.879317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02eacb7-3775-438a-840f-77a91632b7ba None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:34.969474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:34.977495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02eacb7-3775-438a-840f-77a91632b7ba None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:35.010640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11556089-93b0-473e-a6ae-1b17cbdf90dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:35.011309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11556089-93b0-473e-a6ae-1b17cbdf90dc None admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:35.011591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11556089-93b0-473e-a6ae-1b17cbdf90dc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:35.011895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11556089-93b0-473e-a6ae-1b17cbdf90dc None admin] PATH_INFO: `/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/8361f2f9c70b4a3288a187cb2d97d415/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:35.013809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11556089-93b0-473e-a6ae-1b17cbdf90dc None admin] RBAC: Authorizing `identity:create_grant(project_id=b00f8bff707b4491a724e8aaac10c993, user_id=8361f2f9c70b4a3288a187cb2d97d415, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:35.015245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11556089-93b0-473e-a6ae-1b17cbdf90dc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:35.107194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 638/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Sat Aug 6 01:08:34 2022] PUT /identity/v3/projects/b00f8bff707b4491a724e8aaac10c993/users/8361f2f9c70b4a3288a187cb2d97d415/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:08:37.060141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbc757-1b0b-4c56-af3e-993e305068b8 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:37.060756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbc757-1b0b-4c56-af3e-993e305068b8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:37.060756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbc757-1b0b-4c56-af3e-993e305068b8 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:37.062132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 636/1274] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:08:37.067387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fa56e-823f-417a-9c45-4a40ebe6b4d4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:37.067852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fa56e-823f-417a-9c45-4a40ebe6b4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:37.067983 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116fa56e-823f-417a-9c45-4a40ebe6b4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:37.086176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-116fa56e-823f-417a-9c45-4a40ebe6b4d4 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:37.088968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-116fa56e-823f-417a-9c45-4a40ebe6b4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:37.152830 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 639/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:08:37.176276 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbc757-1b0b-4c56-af3e-993e305068b8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:37.277919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbc757-1b0b-4c56-af3e-993e305068b8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:37.295811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:37.313211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b91d9-7984-4ea6-8bd2-b54f1e82d782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:37.313776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b91d9-7984-4ea6-8bd2-b54f1e82d782 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:37.314055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b91d9-7984-4ea6-8bd2-b54f1e82d782 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:37.314327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b91d9-7984-4ea6-8bd2-b54f1e82d782 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:37.323982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b91d9-7984-4ea6-8bd2-b54f1e82d782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-3a6b91d9-7984-4ea6-8bd2-b54f1e82d782 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 01:08:37.329038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:37.330137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:37.331078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 637/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Sat Aug 6 01:08:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:37.337860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fa56e-823f-417a-9c45-4a40ebe6b4d4 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:37.419701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:37.437499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116fa56e-823f-417a-9c45-4a40ebe6b4d4 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:37.469950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88147598-de65-42e3-a640-8fb4ca627af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:37.470566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88147598-de65-42e3-a640-8fb4ca627af3 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:37.470859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88147598-de65-42e3-a640-8fb4ca627af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:37.471141 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88147598-de65-42e3-a640-8fb4ca627af3 None admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:37.471897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88147598-de65-42e3-a640-8fb4ca627af3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:37.472451 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88147598-de65-42e3-a640-8fb4ca627af3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:37.474461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88147598-de65-42e3-a640-8fb4ca627af3 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:37.482812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 640/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Sat Aug 6 01:08:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:37.488664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b91d9-7984-4ea6-8bd2-b54f1e82d782 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:37.587710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b91d9-7984-4ea6-8bd2-b54f1e82d782 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:37.621340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111953b1-42ee-4cc4-af10-b1f7cf9aad1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:37.621967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111953b1-42ee-4cc4-af10-b1f7cf9aad1a None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:37.622248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111953b1-42ee-4cc4-af10-b1f7cf9aad1a None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:37.622524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111953b1-42ee-4cc4-af10-b1f7cf9aad1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:37.633401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111953b1-42ee-4cc4-af10-b1f7cf9aad1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:37.634833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111953b1-42ee-4cc4-af10-b1f7cf9aad1a None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-111953b1-42ee-4cc4-af10-b1f7cf9aad1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:37.648632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:37.654186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:37.655310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:37.655310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:37.655310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:37.655310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:37.655310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 638/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:37.655762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88147598-de65-42e3-a640-8fb4ca627af3 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:37.751604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88147598-de65-42e3-a640-8fb4ca627af3 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:37.785819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0b0ce-5e93-4542-a71a-5e4fcb847adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:37.786491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0b0ce-5e93-4542-a71a-5e4fcb847adc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:37.786771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0b0ce-5e93-4542-a71a-5e4fcb847adc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:37.787051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0b0ce-5e93-4542-a71a-5e4fcb847adc None admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:37.788288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0b0ce-5e93-4542-a71a-5e4fcb847adc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:37.788592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0b0ce-5e93-4542-a71a-5e4fcb847adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:37.789999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0b0ce-5e93-4542-a71a-5e4fcb847adc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:37.802000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 641/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:37.810251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111953b1-42ee-4cc4-af10-b1f7cf9aad1a None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:37.840304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:37.906991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111953b1-42ee-4cc4-af10-b1f7cf9aad1a None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:37.942321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a70ba-0263-48d9-9da6-cc888920c03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:37.942950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a70ba-0263-48d9-9da6-cc888920c03e None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:37.943231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a70ba-0263-48d9-9da6-cc888920c03e None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:37.943507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a70ba-0263-48d9-9da6-cc888920c03e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:37.950505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a70ba-0263-48d9-9da6-cc888920c03e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:37.952007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a70ba-0263-48d9-9da6-cc888920c03e None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-352a70ba-0263-48d9-9da6-cc888920c03e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 01:08:37.963609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:37.964789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:37.965986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 639/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:37.970847 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0b0ce-5e93-4542-a71a-5e4fcb847adc None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:37.974651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:38.067399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0b0ce-5e93-4542-a71a-5e4fcb847adc None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:38.099765 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a642de-9ac8-4209-82c7-43adc306b54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:38.100907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a642de-9ac8-4209-82c7-43adc306b54f None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:38.101186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a642de-9ac8-4209-82c7-43adc306b54f None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:38.101456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a642de-9ac8-4209-82c7-43adc306b54f None admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:38.102235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a642de-9ac8-4209-82c7-43adc306b54f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:38.102538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a642de-9ac8-4209-82c7-43adc306b54f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:38.103987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a642de-9ac8-4209-82c7-43adc306b54f None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:38.122487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 642/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Sat Aug 6 01:08:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:38.128467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a70ba-0263-48d9-9da6-cc888920c03e None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:38.224129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:38.234762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a70ba-0263-48d9-9da6-cc888920c03e None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:38.267425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d3a4c-9a2d-4eb0-9a95-339c9bb14cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:38.268064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d3a4c-9a2d-4eb0-9a95-339c9bb14cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:38.268350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d3a4c-9a2d-4eb0-9a95-339c9bb14cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:38.268626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d3a4c-9a2d-4eb0-9a95-339c9bb14cb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:38.334962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d3a4c-9a2d-4eb0-9a95-339c9bb14cb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:38.336451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d3a4c-9a2d-4eb0-9a95-339c9bb14cb5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:38.423420 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:38.433257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 640/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Sat Aug 6 01:08:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 01:08:38.439235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a642de-9ac8-4209-82c7-43adc306b54f None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:38.498898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:38.538905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a642de-9ac8-4209-82c7-43adc306b54f None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:38.572352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3bce2-7d65-4c0e-bba0-bb3dcd808195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:38.573111 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3bce2-7d65-4c0e-bba0-bb3dcd808195 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:38.573395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3bce2-7d65-4c0e-bba0-bb3dcd808195 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:38.573666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3bce2-7d65-4c0e-bba0-bb3dcd808195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:38.583803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3bce2-7d65-4c0e-bba0-bb3dcd808195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:38.585967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3bce2-7d65-4c0e-bba0-bb3dcd808195 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-bda3bce2-7d65-4c0e-bba0-bb3dcd808195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 01:08:38.600186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:38.601264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:38.602226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 01:08:38.602226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 01:08:38.602226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 01:08:38.602226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:38.602226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 643/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:38.607253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d3a4c-9a2d-4eb0-9a95-339c9bb14cb5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:38.706740 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d3a4c-9a2d-4eb0-9a95-339c9bb14cb5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:38.747905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd78b99-5193-49ac-b233-1e0508deabe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:38.748524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd78b99-5193-49ac-b233-1e0508deabe5 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:38.748806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd78b99-5193-49ac-b233-1e0508deabe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:38.749125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd78b99-5193-49ac-b233-1e0508deabe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:38.749864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd78b99-5193-49ac-b233-1e0508deabe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:38.750267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd78b99-5193-49ac-b233-1e0508deabe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:38.751628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd78b99-5193-49ac-b233-1e0508deabe5 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:38.767628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 641/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Sat Aug 6 01:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:38.768767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3bce2-7d65-4c0e-bba0-bb3dcd808195 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:38.866966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3bce2-7d65-4c0e-bba0-bb3dcd808195 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:38.880231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:38.900789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d1fee-e687-49ff-9b50-0b963489cb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:38.901434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d1fee-e687-49ff-9b50-0b963489cb27 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:38.901716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d1fee-e687-49ff-9b50-0b963489cb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:38.901989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d1fee-e687-49ff-9b50-0b963489cb27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:38.908343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d1fee-e687-49ff-9b50-0b963489cb27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:38.909970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d1fee-e687-49ff-9b50-0b963489cb27 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application [None req-299d1fee-e687-49ff-9b50-0b963489cb27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 01:08:38.920706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:38.921766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:38.922902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 01:08:38.922902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 01:08:38.922902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 01:08:38.922902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 01:08:38.922902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 01:08:38.922902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 01:08:38.922902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 01:08:38.922902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: ERROR keystone.server.flask.application Aug 06 01:08:38.922902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 644/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Sat Aug 6 01:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 01:08:38.927858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd78b99-5193-49ac-b233-1e0508deabe5 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:38.977079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:39.025335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd78b99-5193-49ac-b233-1e0508deabe5 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:39.057233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09619c35-8ef7-4bd0-9755-82da8bb22000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:39.057843 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09619c35-8ef7-4bd0-9755-82da8bb22000 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:39.058126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09619c35-8ef7-4bd0-9755-82da8bb22000 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:39.058398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09619c35-8ef7-4bd0-9755-82da8bb22000 None admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:39.059144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09619c35-8ef7-4bd0-9755-82da8bb22000 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:39.059453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09619c35-8ef7-4bd0-9755-82da8bb22000 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:39.060996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09619c35-8ef7-4bd0-9755-82da8bb22000 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:39.073099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 642/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Sat Aug 6 01:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:39.079193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d1fee-e687-49ff-9b50-0b963489cb27 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:39.177590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d1fee-e687-49ff-9b50-0b963489cb27 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:39.214222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ec832-906f-4351-b6ba-4dd86ae22abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:39.214872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ec832-906f-4351-b6ba-4dd86ae22abc None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:39.215152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ec832-906f-4351-b6ba-4dd86ae22abc None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:39.215427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ec832-906f-4351-b6ba-4dd86ae22abc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:39.216273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ec832-906f-4351-b6ba-4dd86ae22abc None admin] RBAC: Adding query filter params (role.id=933d8f1f9c734bb7beb89f92643c9f56, scope.project.id=b00f8bff707b4491a724e8aaac10c993, user.id=8361f2f9c70b4a3288a187cb2d97d415) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:39.216589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ec832-906f-4351-b6ba-4dd86ae22abc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:39.218079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ec832-906f-4351-b6ba-4dd86ae22abc None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:39.231884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 645/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Sat Aug 6 01:08:39 2022] GET /identity/v3/role_assignments?user.id=8361f2f9c70b4a3288a187cb2d97d415&scope.project.id=b00f8bff707b4491a724e8aaac10c993&role.id=933d8f1f9c734bb7beb89f92643c9f56 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:41.262781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf893-9818-4988-9710-f58e6781db14 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:41.263329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf893-9818-4988-9710-f58e6781db14 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:41.263329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddf893-9818-4988-9710-f58e6781db14 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:41.264699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 643/1288] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:08:41.269822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9b95-aeb9-4922-8e7a-7e2a5c8942b9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:41.270239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9b95-aeb9-4922-8e7a-7e2a5c8942b9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:41.270407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9b95-aeb9-4922-8e7a-7e2a5c8942b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:41.290039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-743e9b95-aeb9-4922-8e7a-7e2a5c8942b9 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:41.291236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-743e9b95-aeb9-4922-8e7a-7e2a5c8942b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:41.319835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:41.355967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 646/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:08:41.377932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddf893-9818-4988-9710-f58e6781db14 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:41.478277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddf893-9818-4988-9710-f58e6781db14 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:41.488561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:41.510756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9335d50-c316-4dbe-8bc0-290f4f6c26bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:41.511383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9335d50-c316-4dbe-8bc0-290f4f6c26bb None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:41.511671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9335d50-c316-4dbe-8bc0-290f4f6c26bb None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:41.511977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9335d50-c316-4dbe-8bc0-290f4f6c26bb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:41.512605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9335d50-c316-4dbe-8bc0-290f4f6c26bb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:41.514010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9335d50-c316-4dbe-8bc0-290f4f6c26bb None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b9335d50-c316-4dbe-8bc0-290f4f6c26bb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:41.523233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:41.524525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:41.525741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 644/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Aug 6 01:08:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:41.530530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e9b95-aeb9-4922-8e7a-7e2a5c8942b9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:41.627021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e9b95-aeb9-4922-8e7a-7e2a5c8942b9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:41.659924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae8e2f-3263-4176-9333-0dde4efdb071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:41.660562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae8e2f-3263-4176-9333-0dde4efdb071 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:41.660839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae8e2f-3263-4176-9333-0dde4efdb071 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:41.661112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae8e2f-3263-4176-9333-0dde4efdb071 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:41.661781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae8e2f-3263-4176-9333-0dde4efdb071 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:41.662081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae8e2f-3263-4176-9333-0dde4efdb071 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:41.664005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae8e2f-3263-4176-9333-0dde4efdb071 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:41.668718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 647/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Sat Aug 6 01:08:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:41.674622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9335d50-c316-4dbe-8bc0-290f4f6c26bb None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:41.772354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9335d50-c316-4dbe-8bc0-290f4f6c26bb None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:41.805230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857f222-034b-43a7-b3da-f7dbd5f6fa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:41.805834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857f222-034b-43a7-b3da-f7dbd5f6fa38 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:41.806118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857f222-034b-43a7-b3da-f7dbd5f6fa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:41.806395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857f222-034b-43a7-b3da-f7dbd5f6fa38 None admin] PATH_INFO: `/v3/services` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:41.807071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857f222-034b-43a7-b3da-f7dbd5f6fa38 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:41.807380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857f222-034b-43a7-b3da-f7dbd5f6fa38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:41.808776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857f222-034b-43a7-b3da-f7dbd5f6fa38 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:41.813107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 645/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Sat Aug 6 01:08:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:43.817306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f37aa-f405-400e-b109-c4092d76abf8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:43.817849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f37aa-f405-400e-b109-c4092d76abf8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:43.817849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f37aa-f405-400e-b109-c4092d76abf8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:43.819703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 648/1293] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:08:43.824967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5948c-983f-4dcf-bca3-29fad51f6fe1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:43.825338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5948c-983f-4dcf-bca3-29fad51f6fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:43.826621 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5948c-983f-4dcf-bca3-29fad51f6fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:43.846108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c9f5948c-983f-4dcf-bca3-29fad51f6fe1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:43.846108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c9f5948c-983f-4dcf-bca3-29fad51f6fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:43.874437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:43.913053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 646/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:08:43.935142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f37aa-f405-400e-b109-c4092d76abf8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:43.966550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:44.032628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f37aa-f405-400e-b109-c4092d76abf8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:44.065074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd2c67-d69f-4e54-9706-509ab622a33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:44.065683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd2c67-d69f-4e54-9706-509ab622a33b None admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:44.075635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd2c67-d69f-4e54-9706-509ab622a33b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:44.075635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd2c67-d69f-4e54-9706-509ab622a33b None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:44.075635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd2c67-d69f-4e54-9706-509ab622a33b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:44.075635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd2c67-d69f-4e54-9706-509ab622a33b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:44.153505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 649/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Sat Aug 6 01:08:43 2022] POST /identity/v3/services => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:08:46.144368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7abda-0d4f-4144-8824-4d10619aaf9a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:46.144943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7abda-0d4f-4144-8824-4d10619aaf9a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:46.144943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7abda-0d4f-4144-8824-4d10619aaf9a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:46.146247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 647/1296] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:08:46.152170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f106c-30ee-4990-805f-55ba02bb3bf2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:46.152596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f106c-30ee-4990-805f-55ba02bb3bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:46.157755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f106c-30ee-4990-805f-55ba02bb3bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:46.170807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-af9f106c-30ee-4990-805f-55ba02bb3bf2 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:46.172017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-af9f106c-30ee-4990-805f-55ba02bb3bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:46.243399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 650/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:08:46.265359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7abda-0d4f-4144-8824-4d10619aaf9a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:46.362982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7abda-0d4f-4144-8824-4d10619aaf9a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:46.395933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71466b1-d950-4d97-a522-f319af1918ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:46.396519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71466b1-d950-4d97-a522-f319af1918ab None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:46.396799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71466b1-d950-4d97-a522-f319af1918ab None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:46.397079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71466b1-d950-4d97-a522-f319af1918ab None admin] PATH_INFO: `/v3/services/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:46.397704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71466b1-d950-4d97-a522-f319af1918ab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:46.399052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71466b1-d950-4d97-a522-f319af1918ab None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-b71466b1-d950-4d97-a522-f319af1918ab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:46.407868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:46.408882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:46.409972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 648/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Aug 6 01:08:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:46.415582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f106c-30ee-4990-805f-55ba02bb3bf2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:46.434296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:46.510994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f106c-30ee-4990-805f-55ba02bb3bf2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:46.544764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037eef0-8887-42fd-a697-21ed722adf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:46.545383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037eef0-8887-42fd-a697-21ed722adf95 None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:46.545670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037eef0-8887-42fd-a697-21ed722adf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:46.545948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037eef0-8887-42fd-a697-21ed722adf95 None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:46.546676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037eef0-8887-42fd-a697-21ed722adf95 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:46.546984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037eef0-8887-42fd-a697-21ed722adf95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:46.548381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037eef0-8887-42fd-a697-21ed722adf95 None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:46.553393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 651/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Sat Aug 6 01:08:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:46.559650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71466b1-d950-4d97-a522-f319af1918ab None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:46.590980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:46.658062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71466b1-d950-4d97-a522-f319af1918ab None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:46.691762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5bcca-7ed4-4491-8e2e-fdd88bbb350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:46.692405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5bcca-7ed4-4491-8e2e-fdd88bbb350b None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:46.692689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5bcca-7ed4-4491-8e2e-fdd88bbb350b None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:46.692967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5bcca-7ed4-4491-8e2e-fdd88bbb350b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:46.693722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5bcca-7ed4-4491-8e2e-fdd88bbb350b None admin] RBAC: Adding query filter params (interface=public, service_id=9a2797ac68874c319625a1380c5c2c45, region_id=RegionOne) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:46.694084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5bcca-7ed4-4491-8e2e-fdd88bbb350b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:46.695443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5bcca-7ed4-4491-8e2e-fdd88bbb350b None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:46.700424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 649/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Sat Aug 6 01:08:46 2022] GET /identity/v3/endpoints?service_id=9a2797ac68874c319625a1380c5c2c45&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:46.706132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037eef0-8887-42fd-a697-21ed722adf95 None admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:46.784985 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:46.803235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037eef0-8887-42fd-a697-21ed722adf95 None admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:46.837903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fd1eb-eec4-4735-8258-031da7ffad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:46.838525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd1eb-eec4-4735-8258-031da7ffad2b None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:46.838805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd1eb-eec4-4735-8258-031da7ffad2b None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:46.839086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd1eb-eec4-4735-8258-031da7ffad2b None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:46.839735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fd1eb-eec4-4735-8258-031da7ffad2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:46.841138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fd1eb-eec4-4735-8258-031da7ffad2b None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:46.847841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 652/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Sat Aug 6 01:08:46 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:08:48.866303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a029ecf-56cf-40e5-b1cc-5bc8256818a5 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:48.866801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a029ecf-56cf-40e5-b1cc-5bc8256818a5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:48.866801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a029ecf-56cf-40e5-b1cc-5bc8256818a5 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:48.868272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 650/1302] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:08:48.873422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81b099-2921-426b-9c48-ceba25c846a1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:48.873861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81b099-2921-426b-9c48-ceba25c846a1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:48.873977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81b099-2921-426b-9c48-ceba25c846a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:48.891692 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6e81b099-2921-426b-9c48-ceba25c846a1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:08:48.892912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6e81b099-2921-426b-9c48-ceba25c846a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:08:48.956653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 653/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:08:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 01:08:48.980759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a029ecf-56cf-40e5-b1cc-5bc8256818a5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:49.072518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:49.079000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a029ecf-56cf-40e5-b1cc-5bc8256818a5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:49.111767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cfc67-63b1-494d-9f07-c719c8e76386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:49.112423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cfc67-63b1-494d-9f07-c719c8e76386 None admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:49.112708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cfc67-63b1-494d-9f07-c719c8e76386 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:49.112986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cfc67-63b1-494d-9f07-c719c8e76386 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:49.113609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cfc67-63b1-494d-9f07-c719c8e76386 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:49.114965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cfc67-63b1-494d-9f07-c719c8e76386 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application [None req-6c0cfc67-63b1-494d-9f07-c719c8e76386 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 01:08:49.124358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 01:08:49.125549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application with Lock( Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self._enter() Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application return self.creator() Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application created_value = creator( Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: ERROR keystone.server.flask.application Aug 06 01:08:49.126693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 651/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Sat Aug 6 01:08:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 01:08:49.132165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81b099-2921-426b-9c48-ceba25c846a1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:49.206267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:49.228524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81b099-2921-426b-9c48-ceba25c846a1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:49.260896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e76ed-e0f5-4995-9dc9-2c3c1f17683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:49.261585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e76ed-e0f5-4995-9dc9-2c3c1f17683c None admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:49.261880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e76ed-e0f5-4995-9dc9-2c3c1f17683c None admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:49.262147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e76ed-e0f5-4995-9dc9-2c3c1f17683c None admin] PATH_INFO: `/v3/services` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:49.262835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e76ed-e0f5-4995-9dc9-2c3c1f17683c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:08:49.263147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e76ed-e0f5-4995-9dc9-2c3c1f17683c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:49.264564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e76ed-e0f5-4995-9dc9-2c3c1f17683c None admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:49.269595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 654/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Sat Aug 6 01:08:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:08:49.275299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cfc67-63b1-494d-9f07-c719c8e76386 None admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:08:49.378546 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cfc67-63b1-494d-9f07-c719c8e76386 None admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:08:49.410515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aec231-c367-4309-96f6-ea4eb3d99a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:08:49.411136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec231-c367-4309-96f6-ea4eb3d99a74 None admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:08:49.411403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec231-c367-4309-96f6-ea4eb3d99a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:08:49.411678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec231-c367-4309-96f6-ea4eb3d99a74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:08:49.412929 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aec231-c367-4309-96f6-ea4eb3d99a74 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:08:49.414273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aec231-c367-4309-96f6-ea4eb3d99a74 None admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:08:49.518482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 652/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:08:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:08:59.283419 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=30 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:08:59.513731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:34.170731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8c8c-39a6-4b03-a8de-474973239f6c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:34.171294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8c8c-39a6-4b03-a8de-474973239f6c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:34.171294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8c8c-39a6-4b03-a8de-474973239f6c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:34.178770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585c983-c7a8-4cbc-8968-8ff8a4113455 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:34.179276 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585c983-c7a8-4cbc-8968-8ff8a4113455 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:34.179371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585c983-c7a8-4cbc-8968-8ff8a4113455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:34.181050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 655/1307] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:14:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:14:34.267032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9585c983-c7a8-4cbc-8968-8ff8a4113455 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:34.268331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9585c983-c7a8-4cbc-8968-8ff8a4113455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:34.408853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 653/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:14:34.437780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac2d92-bb81-4af2-bbf7-c6b39dcacf3e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:34.438357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac2d92-bb81-4af2-bbf7-c6b39dcacf3e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:34.438534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac2d92-bb81-4af2-bbf7-c6b39dcacf3e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:34.440183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 656/1309] 10.222.0.25 () {54 vars in 965 bytes} [Sat Aug 6 01:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:14:34.446464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccf0ff-94f7-4349-992a-3a4748748f2a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:34.446754 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccf0ff-94f7-4349-992a-3a4748748f2a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:34.447038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccf0ff-94f7-4349-992a-3a4748748f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:34.565114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-01ccf0ff-94f7-4349-992a-3a4748748f2a None None] MFA Rules not processed for user `2c14fb133a0f419390fb0d00c600bdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:34.566442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-01ccf0ff-94f7-4349-992a-3a4748748f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:34.592191 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:34.653098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 654/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Sat Aug 6 01:14:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:14:34.662576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac2d92-bb81-4af2-bbf7-c6b39dcacf3e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:14:34.664317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-96ac2d92-bb81-4af2-bbf7-c6b39dcacf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:34.774566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac2d92-bb81-4af2-bbf7-c6b39dcacf3e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:14:34.852770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4e4eb-331f-4e0d-9eba-3200ca3f846f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:14:34.853395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4e4eb-331f-4e0d-9eba-3200ca3f846f service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:34.853674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4e4eb-331f-4e0d-9eba-3200ca3f846f service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:34.853945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4e4eb-331f-4e0d-9eba-3200ca3f846f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:34.855543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7cd4e4eb-331f-4e0d-9eba-3200ca3f846f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:34.910924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:34.966432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4e4eb-331f-4e0d-9eba-3200ca3f846f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:14:34.967897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4e4eb-331f-4e0d-9eba-3200ca3f846f service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:14:34.986681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 657/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:14:37.283518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ef47-3d09-4d91-a66f-911ae16f18bd None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:37.284089 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ef47-3d09-4d91-a66f-911ae16f18bd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:37.284089 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8ef47-3d09-4d91-a66f-911ae16f18bd None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:37.286030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 655/1312] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:14:37.291124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e5009-4612-4a3a-9915-8be40e5a6213 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:37.291571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e5009-4612-4a3a-9915-8be40e5a6213 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:37.291703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e5009-4612-4a3a-9915-8be40e5a6213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:37.363161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2b5e5009-4612-4a3a-9915-8be40e5a6213 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:37.364419 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2b5e5009-4612-4a3a-9915-8be40e5a6213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:37.466961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 658/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:14:37.506309 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8ef47-3d09-4d91-a66f-911ae16f18bd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:14:37.507959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6eb8ef47-3d09-4d91-a66f-911ae16f18bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:37.613921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8ef47-3d09-4d91-a66f-911ae16f18bd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:14:37.684408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:37.687226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583bce6-8ed0-4322-94db-dcacd393831b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:14:37.687900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583bce6-8ed0-4322-94db-dcacd393831b service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:37.688216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583bce6-8ed0-4322-94db-dcacd393831b service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:37.688492 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583bce6-8ed0-4322-94db-dcacd393831b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:37.690205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8583bce6-8ed0-4322-94db-dcacd393831b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:37.801647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583bce6-8ed0-4322-94db-dcacd393831b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:14:37.803119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583bce6-8ed0-4322-94db-dcacd393831b service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:14:37.821512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 656/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:14:40.360754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919cc59-9bc2-47c8-8cf0-8881c2ebcf0b None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:40.361284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919cc59-9bc2-47c8-8cf0-8881c2ebcf0b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:40.361379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919cc59-9bc2-47c8-8cf0-8881c2ebcf0b None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:40.362680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 659/1315] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:14:40.368343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645fe3b-14db-48eb-99d5-d02c3c127e4f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:40.368729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645fe3b-14db-48eb-99d5-d02c3c127e4f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:40.368913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645fe3b-14db-48eb-99d5-d02c3c127e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:40.434072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f645fe3b-14db-48eb-99d5-d02c3c127e4f None None] MFA Rules not processed for user `a5948add9ba0485a99f1629cd6f9962b`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:40.435282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f645fe3b-14db-48eb-99d5-d02c3c127e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:40.488082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:40.542658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 657/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:14:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:14:40.598104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b0c75-6e07-40c7-8a83-ecfea16b1dab None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:40.598522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b0c75-6e07-40c7-8a83-ecfea16b1dab None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:40.598801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b0c75-6e07-40c7-8a83-ecfea16b1dab None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:40.600224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 660/1317] 10.222.0.25 () {54 vars in 965 bytes} [Sat Aug 6 01:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:14:40.606939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f66f9a-84a4-48d2-9010-2432add57d44 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:40.607272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f66f9a-84a4-48d2-9010-2432add57d44 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:40.607551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f66f9a-84a4-48d2-9010-2432add57d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:40.681495 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-04f66f9a-84a4-48d2-9010-2432add57d44 None None] MFA Rules not processed for user `2c14fb133a0f419390fb0d00c600bdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:40.682692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-04f66f9a-84a4-48d2-9010-2432add57d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:40.755413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 658/1318] 10.222.0.25 () {58 vars in 1062 bytes} [Sat Aug 6 01:14:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:14:40.764157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b0c75-6e07-40c7-8a83-ecfea16b1dab None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:14:40.765725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-429b0c75-6e07-40c7-8a83-ecfea16b1dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:40.775247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:40.867689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b0c75-6e07-40c7-8a83-ecfea16b1dab None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:14:40.941904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7025b50-9b46-4a83-b8ad-0574dfee1fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:14:40.942525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7025b50-9b46-4a83-b8ad-0574dfee1fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:40.942799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7025b50-9b46-4a83-b8ad-0574dfee1fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:40.943066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7025b50-9b46-4a83-b8ad-0574dfee1fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:40.944705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d7025b50-9b46-4a83-b8ad-0574dfee1fd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:41.004946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:41.049937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7025b50-9b46-4a83-b8ad-0574dfee1fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:14:41.051359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7025b50-9b46-4a83-b8ad-0574dfee1fd6 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:14:41.068477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 661/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:14:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:14:44.279272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a41e38-e813-4e28-98c4-9793f1496b4a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:44.279860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a41e38-e813-4e28-98c4-9793f1496b4a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:44.279860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a41e38-e813-4e28-98c4-9793f1496b4a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:44.281220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 659/1320] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:14:44.286925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cec1c3-cc0e-4519-ad95-cc9ec0976717 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:44.287347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cec1c3-cc0e-4519-ad95-cc9ec0976717 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:44.287483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cec1c3-cc0e-4519-ad95-cc9ec0976717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:44.352113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-21cec1c3-cc0e-4519-ad95-cc9ec0976717 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:44.353319 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-21cec1c3-cc0e-4519-ad95-cc9ec0976717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:44.446980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 662/1321] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:14:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:14:44.486250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a41e38-e813-4e28-98c4-9793f1496b4a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:14:44.487807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-30a41e38-e813-4e28-98c4-9793f1496b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:44.537863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:44.594260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a41e38-e813-4e28-98c4-9793f1496b4a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:14:44.672235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102904df-d36a-4829-b631-db0cdaa11e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:14:44.672857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102904df-d36a-4829-b631-db0cdaa11e14 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:44.673142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102904df-d36a-4829-b631-db0cdaa11e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:44.673426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102904df-d36a-4829-b631-db0cdaa11e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:44.675053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-102904df-d36a-4829-b631-db0cdaa11e14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:44.768028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:44.781390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102904df-d36a-4829-b631-db0cdaa11e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:14:44.782824 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102904df-d36a-4829-b631-db0cdaa11e14 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:14:44.800900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 660/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:14:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:14:47.872888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9857e-d294-4312-af64-49cd37c8fb76 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:47.873435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9857e-d294-4312-af64-49cd37c8fb76 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:47.873435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9857e-d294-4312-af64-49cd37c8fb76 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:47.874907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 663/1323] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:14:47.880571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606979c-5957-474a-9e7e-7709783d2c05 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:47.880974 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606979c-5957-474a-9e7e-7709783d2c05 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:47.881130 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606979c-5957-474a-9e7e-7709783d2c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:47.947193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3606979c-5957-474a-9e7e-7709783d2c05 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:47.948442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3606979c-5957-474a-9e7e-7709783d2c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:48.047727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 661/1324] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:14:48.084512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9857e-d294-4312-af64-49cd37c8fb76 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:14:48.086066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-00f9857e-d294-4312-af64-49cd37c8fb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:48.107069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:48.196621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9857e-d294-4312-af64-49cd37c8fb76 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:14:48.273737 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660cb5ab-e63f-4d88-8585-b8c95ad3a0f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:14:48.274436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cb5ab-e63f-4d88-8585-b8c95ad3a0f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:48.274722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cb5ab-e63f-4d88-8585-b8c95ad3a0f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:48.275012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cb5ab-e63f-4d88-8585-b8c95ad3a0f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:48.276664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-660cb5ab-e63f-4d88-8585-b8c95ad3a0f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:48.352806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:48.389049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660cb5ab-e63f-4d88-8585-b8c95ad3a0f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:14:48.390491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660cb5ab-e63f-4d88-8585-b8c95ad3a0f4 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:14:48.408704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 664/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:14:50.461409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a9513-8d01-4a66-9ce7-a59ec6245dc6 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:50.461942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a9513-8d01-4a66-9ce7-a59ec6245dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:50.461942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a9513-8d01-4a66-9ce7-a59ec6245dc6 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:50.463309 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 662/1326] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:14:50.468769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb747a-3d5a-48c5-acca-9354c57083a8 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:50.469193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb747a-3d5a-48c5-acca-9354c57083a8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:50.469353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb747a-3d5a-48c5-acca-9354c57083a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:50.536394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8bfb747a-3d5a-48c5-acca-9354c57083a8 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:50.537495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8bfb747a-3d5a-48c5-acca-9354c57083a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:50.641743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 665/1327] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:14:50.678383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a9513-8d01-4a66-9ce7-a59ec6245dc6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:14:50.690137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-703a9513-8d01-4a66-9ce7-a59ec6245dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:50.740156 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:50.783528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a9513-8d01-4a66-9ce7-a59ec6245dc6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:14:50.856546 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d443a4b-23d2-4775-87c8-b81dab553527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:14:50.857084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d443a4b-23d2-4775-87c8-b81dab553527 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:50.857368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d443a4b-23d2-4775-87c8-b81dab553527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:50.857643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d443a4b-23d2-4775-87c8-b81dab553527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:50.859294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9d443a4b-23d2-4775-87c8-b81dab553527 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:50.969986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:50.972196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d443a4b-23d2-4775-87c8-b81dab553527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:14:50.974345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d443a4b-23d2-4775-87c8-b81dab553527 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:14:50.992354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 663/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:14:53.051846 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06b72-2db0-4d22-b3d1-5235f7c9a835 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:53.052516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06b72-2db0-4d22-b3d1-5235f7c9a835 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:53.052516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06b72-2db0-4d22-b3d1-5235f7c9a835 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:53.053795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 666/1329] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:14:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:14:53.059693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704a80e-6535-498c-8410-d341b4a8ee56 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:53.060265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704a80e-6535-498c-8410-d341b4a8ee56 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:53.060265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704a80e-6535-498c-8410-d341b4a8ee56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:53.127250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7704a80e-6535-498c-8410-d341b4a8ee56 None None] MFA Rules not processed for user `a5948add9ba0485a99f1629cd6f9962b`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:53.128475 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7704a80e-6535-498c-8410-d341b4a8ee56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:53.226150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 664/1330] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:14:53 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:14:53.264039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef06b72-2db0-4d22-b3d1-5235f7c9a835 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:14:53.265603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1ef06b72-2db0-4d22-b3d1-5235f7c9a835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:53.297094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:53.374557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef06b72-2db0-4d22-b3d1-5235f7c9a835 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:14:53.448791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac142b-a3de-41bd-8881-e4aff5a4fe29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:14:53.449422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac142b-a3de-41bd-8881-e4aff5a4fe29 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:53.449703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac142b-a3de-41bd-8881-e4aff5a4fe29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:53.449979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac142b-a3de-41bd-8881-e4aff5a4fe29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:53.451591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4fac142b-a3de-41bd-8881-e4aff5a4fe29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:53.530743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:53.562315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac142b-a3de-41bd-8881-e4aff5a4fe29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:14:53.563805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac142b-a3de-41bd-8881-e4aff5a4fe29 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:14:53.585849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 667/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:14:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:14:56.071898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f72f7c-8817-466c-a250-f2ee217989d1 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:56.072402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f72f7c-8817-466c-a250-f2ee217989d1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:56.072402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f72f7c-8817-466c-a250-f2ee217989d1 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:56.073747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 665/1332] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:14:56.078804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b68f2-5934-4e33-8a72-3065f1ada902 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:56.079197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b68f2-5934-4e33-8a72-3065f1ada902 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:56.079357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b68f2-5934-4e33-8a72-3065f1ada902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:56.146770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-003b68f2-5934-4e33-8a72-3065f1ada902 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:56.148007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-003b68f2-5934-4e33-8a72-3065f1ada902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:56.241525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 668/1333] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:14:56.279954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f72f7c-8817-466c-a250-f2ee217989d1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:14:56.281478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-97f72f7c-8817-466c-a250-f2ee217989d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:56.349345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:56.382363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f72f7c-8817-466c-a250-f2ee217989d1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:14:56.452400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff8f4a-e3b3-44f6-8b09-5548ea398a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:14:56.452999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff8f4a-e3b3-44f6-8b09-5548ea398a45 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:56.453278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff8f4a-e3b3-44f6-8b09-5548ea398a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:56.453610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff8f4a-e3b3-44f6-8b09-5548ea398a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:56.455203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3eff8f4a-e3b3-44f6-8b09-5548ea398a45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:56.561509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff8f4a-e3b3-44f6-8b09-5548ea398a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:14:56.562931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff8f4a-e3b3-44f6-8b09-5548ea398a45 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:14:56.574957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:56.584480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 666/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:14:58.629996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5dd0a-2498-4e07-8624-39f5cdc0e3fe None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:58.630532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5dd0a-2498-4e07-8624-39f5cdc0e3fe None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:58.630532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5dd0a-2498-4e07-8624-39f5cdc0e3fe None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:58.631965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 669/1335] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:14:58.637162 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcb8d9-8e0e-476e-aa3d-12753a9b0430 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:58.637586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcb8d9-8e0e-476e-aa3d-12753a9b0430 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:58.637710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcb8d9-8e0e-476e-aa3d-12753a9b0430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:58.711230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-05bcb8d9-8e0e-476e-aa3d-12753a9b0430 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:14:58.712472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-05bcb8d9-8e0e-476e-aa3d-12753a9b0430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:58.805016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 667/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:14:58.845974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5dd0a-2498-4e07-8624-39f5cdc0e3fe None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:14:58.847494 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b3e5dd0a-2498-4e07-8624-39f5cdc0e3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:58.886746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:58.947384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5dd0a-2498-4e07-8624-39f5cdc0e3fe None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:14:59.016797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2083f-3efb-4d57-8d61-5d0a00f96aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:14:59.017408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2083f-3efb-4d57-8d61-5d0a00f96aab service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:14:59.017686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2083f-3efb-4d57-8d61-5d0a00f96aab service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:14:59.017956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2083f-3efb-4d57-8d61-5d0a00f96aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:14:59.020595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d7c2083f-3efb-4d57-8d61-5d0a00f96aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:14:59.099935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:14:59.125708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2083f-3efb-4d57-8d61-5d0a00f96aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:14:59.127155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2083f-3efb-4d57-8d61-5d0a00f96aab service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:14:59.145655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 670/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:01.826948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd408d-2cbc-46f6-a478-78c48b9461c4 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:01.827368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd408d-2cbc-46f6-a478-78c48b9461c4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:01.827505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd408d-2cbc-46f6-a478-78c48b9461c4 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:01.828892 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 668/1338] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:01.834736 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4af4d2-cdcb-43a6-be86-28a19ed0745e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:01.835183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4af4d2-cdcb-43a6-be86-28a19ed0745e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:01.835290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4af4d2-cdcb-43a6-be86-28a19ed0745e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:01.899756 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7c4af4d2-cdcb-43a6-be86-28a19ed0745e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:01.900999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7c4af4d2-cdcb-43a6-be86-28a19ed0745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:01.995022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 671/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:15:02.031322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd408d-2cbc-46f6-a478-78c48b9461c4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:02.032872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6fdd408d-2cbc-46f6-a478-78c48b9461c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:02.106652 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:02.133911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd408d-2cbc-46f6-a478-78c48b9461c4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:02.206028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d2815-1638-4788-97ce-cb747072b724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:02.207177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d2815-1638-4788-97ce-cb747072b724 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:02.207460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d2815-1638-4788-97ce-cb747072b724 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:02.207735 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d2815-1638-4788-97ce-cb747072b724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:02.209651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8e0d2815-1638-4788-97ce-cb747072b724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:02.314589 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d2815-1638-4788-97ce-cb747072b724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:02.316584 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d2815-1638-4788-97ce-cb747072b724 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:02.334998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 669/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:06.459600 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d0dba-761e-4ce9-a68b-2b19cc60776d None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:06.460127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d0dba-761e-4ce9-a68b-2b19cc60776d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:06.460220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d0dba-761e-4ce9-a68b-2b19cc60776d None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:06.461558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 672/1341] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:06.467033 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6811fba-db2f-4880-8ebb-579fbb027cda None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:06.467491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6811fba-db2f-4880-8ebb-579fbb027cda None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:06.467590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6811fba-db2f-4880-8ebb-579fbb027cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:06.473119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:06.535376 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c6811fba-db2f-4880-8ebb-579fbb027cda None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:06.536627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c6811fba-db2f-4880-8ebb-579fbb027cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:06.642011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 670/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:06.674923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d0dba-761e-4ce9-a68b-2b19cc60776d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:06.676599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c67d0dba-761e-4ce9-a68b-2b19cc60776d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:06.725359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:06.784400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d0dba-761e-4ce9-a68b-2b19cc60776d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:06.856884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d50192-41c1-4d12-a700-cb5dc40d7e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:06.857617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d50192-41c1-4d12-a700-cb5dc40d7e59 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:06.857907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d50192-41c1-4d12-a700-cb5dc40d7e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:06.858186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d50192-41c1-4d12-a700-cb5dc40d7e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:06.859896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e7d50192-41c1-4d12-a700-cb5dc40d7e59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:06.950821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:06.966822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d50192-41c1-4d12-a700-cb5dc40d7e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:06.968321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d50192-41c1-4d12-a700-cb5dc40d7e59 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:06.987142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 673/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:10.186552 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97df731-a3a4-4bf4-99ca-c6a410383d38 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:10.187075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97df731-a3a4-4bf4-99ca-c6a410383d38 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:10.187075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97df731-a3a4-4bf4-99ca-c6a410383d38 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:10.188403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 671/1344] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:10.193643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4ed23-32f2-4483-b6c4-64731cc1b1ac None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:10.194068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4ed23-32f2-4483-b6c4-64731cc1b1ac None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:10.194190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4ed23-32f2-4483-b6c4-64731cc1b1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:10.259595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6dd4ed23-32f2-4483-b6c4-64731cc1b1ac None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:10.260819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6dd4ed23-32f2-4483-b6c4-64731cc1b1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:10.361324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 674/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:10.399742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97df731-a3a4-4bf4-99ca-c6a410383d38 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:10.401336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a97df731-a3a4-4bf4-99ca-c6a410383d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:10.488418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:10.505327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97df731-a3a4-4bf4-99ca-c6a410383d38 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:10.578647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4019f-682e-41e0-aedf-78fd24f75d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:10.579565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4019f-682e-41e0-aedf-78fd24f75d92 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:10.580046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4019f-682e-41e0-aedf-78fd24f75d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:10.580472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4019f-682e-41e0-aedf-78fd24f75d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:10.582915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f1a4019f-682e-41e0-aedf-78fd24f75d92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:10.704566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4019f-682e-41e0-aedf-78fd24f75d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:10.705910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4019f-682e-41e0-aedf-78fd24f75d92 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:10.727351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 672/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:15.234711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab920d-e482-457b-9cdb-aa2aee947f66 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:15.235226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab920d-e482-457b-9cdb-aa2aee947f66 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:15.235345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab920d-e482-457b-9cdb-aa2aee947f66 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:15.236808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 675/1347] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:15.242554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b6dd2-9b84-4baf-bb32-706bafb05926 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:15.243034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b6dd2-9b84-4baf-bb32-706bafb05926 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:15.243134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b6dd2-9b84-4baf-bb32-706bafb05926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:15.258851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:15.312990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d24b6dd2-9b84-4baf-bb32-706bafb05926 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:15.314223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d24b6dd2-9b84-4baf-bb32-706bafb05926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:15.417149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 673/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:15.455917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab920d-e482-457b-9cdb-aa2aee947f66 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:15.457590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-00ab920d-e482-457b-9cdb-aa2aee947f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:15.516965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:15.565348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab920d-e482-457b-9cdb-aa2aee947f66 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:15.643159 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2afeb-4905-48b4-9096-e04e55abcfbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:15.643711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2afeb-4905-48b4-9096-e04e55abcfbc service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:15.644081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2afeb-4905-48b4-9096-e04e55abcfbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:15.644340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2afeb-4905-48b4-9096-e04e55abcfbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:15.645934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2ca2afeb-4905-48b4-9096-e04e55abcfbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:15.755264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:15.759343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2afeb-4905-48b4-9096-e04e55abcfbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:15.761223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2afeb-4905-48b4-9096-e04e55abcfbc service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:15.782306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 676/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:20.689473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee422f-0fa5-4a4e-b452-2d406572b118 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:20.690437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee422f-0fa5-4a4e-b452-2d406572b118 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:20.690437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee422f-0fa5-4a4e-b452-2d406572b118 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:20.691914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 674/1350] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:20.697068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cacca-6209-46f9-8a9c-9fed2cd8f587 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:20.697476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cacca-6209-46f9-8a9c-9fed2cd8f587 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:20.697683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cacca-6209-46f9-8a9c-9fed2cd8f587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:20.845161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-308cacca-6209-46f9-8a9c-9fed2cd8f587 None None] MFA Rules not processed for user `8361f2f9c70b4a3288a187cb2d97d415`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:20.846390 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-308cacca-6209-46f9-8a9c-9fed2cd8f587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:20.973335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 677/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:15:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:15:25.050202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f162e8-0377-4075-8f82-196a918c279e None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:25.051075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f162e8-0377-4075-8f82-196a918c279e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:25.051546 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f162e8-0377-4075-8f82-196a918c279e None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:25.052924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 675/1352] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:25.058794 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3c258-4758-4fbf-8b4b-3519edbbd2ac None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:25.059215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3c258-4758-4fbf-8b4b-3519edbbd2ac None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:25.059349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3c258-4758-4fbf-8b4b-3519edbbd2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:25.117197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:25.140066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f4b3c258-4758-4fbf-8b4b-3519edbbd2ac None None] MFA Rules not processed for user `8361f2f9c70b4a3288a187cb2d97d415`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:25.141281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f4b3c258-4758-4fbf-8b4b-3519edbbd2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:25.252342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 678/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:15:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:25.415022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=59 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:26.623656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78961107-9c3a-401f-897e-3b451f797920 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:26.624299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78961107-9c3a-401f-897e-3b451f797920 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:26.624299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78961107-9c3a-401f-897e-3b451f797920 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:26.625601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 676/1354] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:26.630720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ed7e3-dbb6-4b1a-a257-f28f2b69343a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:26.631136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ed7e3-dbb6-4b1a-a257-f28f2b69343a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:26.631270 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ed7e3-dbb6-4b1a-a257-f28f2b69343a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:26.708548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-459ed7e3-dbb6-4b1a-a257-f28f2b69343a None None] MFA Rules not processed for user `8361f2f9c70b4a3288a187cb2d97d415`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:26.709764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-459ed7e3-dbb6-4b1a-a257-f28f2b69343a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:26.751011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:26.813839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 679/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:15:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:30.322810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883151a9-fc52-47e7-a608-77d508ac788b None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:30.323628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883151a9-fc52-47e7-a608-77d508ac788b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:30.325976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883151a9-fc52-47e7-a608-77d508ac788b None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:30.330268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 677/1356] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:30.333764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805cf5f-f16d-44c6-b8be-a0d1e2243f76 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:30.334642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805cf5f-f16d-44c6-b8be-a0d1e2243f76 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:30.334925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805cf5f-f16d-44c6-b8be-a0d1e2243f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:30.413808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7805cf5f-f16d-44c6-b8be-a0d1e2243f76 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:30.415325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7805cf5f-f16d-44c6-b8be-a0d1e2243f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:30.523936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 680/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:30.588790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2b2d6-39e1-4abd-b0a7-f18f4429ede8 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:30.594672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2b2d6-39e1-4abd-b0a7-f18f4429ede8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:30.594672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2b2d6-39e1-4abd-b0a7-f18f4429ede8 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:30.594672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 678/1358] 10.222.0.25 () {54 vars in 962 bytes} [Sat Aug 6 01:15:30 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:30.598092 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5adee3-9c41-414a-a920-c3c492188447 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:30.598455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5adee3-9c41-414a-a920-c3c492188447 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:30.598800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5adee3-9c41-414a-a920-c3c492188447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:30.621156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:30.680318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1e5adee3-9c41-414a-a920-c3c492188447 None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:30.681677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1e5adee3-9c41-414a-a920-c3c492188447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:30.805563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 681/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Sat Aug 6 01:15:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:30.813446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2b2d6-39e1-4abd-b0a7-f18f4429ede8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:30.815013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-06f2b2d6-39e1-4abd-b0a7-f18f4429ede8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:30.962911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2b2d6-39e1-4abd-b0a7-f18f4429ede8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:31.037561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:31.079090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5803e9e-58ab-4aca-8b73-41ba0ca05723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:31.079615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5803e9e-58ab-4aca-8b73-41ba0ca05723 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:31.079925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5803e9e-58ab-4aca-8b73-41ba0ca05723 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:31.080201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5803e9e-58ab-4aca-8b73-41ba0ca05723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:31.081890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d5803e9e-58ab-4aca-8b73-41ba0ca05723 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:31.188336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5803e9e-58ab-4aca-8b73-41ba0ca05723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:31.189977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5803e9e-58ab-4aca-8b73-41ba0ca05723 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:31.211544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 679/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:33.425685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841e77c-3691-4418-befc-90080da2feb9 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:33.426201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841e77c-3691-4418-befc-90080da2feb9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:33.426201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841e77c-3691-4418-befc-90080da2feb9 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:33.427758 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 682/1361] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:33.432801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017db85-b034-4b7e-b919-5dabb62d4eaa None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:33.433176 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017db85-b034-4b7e-b919-5dabb62d4eaa None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:33.433372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017db85-b034-4b7e-b919-5dabb62d4eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:33.503440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7017db85-b034-4b7e-b919-5dabb62d4eaa None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:33.504687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7017db85-b034-4b7e-b919-5dabb62d4eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:33.544182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:33.615680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 680/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:33.662065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841e77c-3691-4418-befc-90080da2feb9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:33.663558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e841e77c-3691-4418-befc-90080da2feb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:33.735711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:33.829247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841e77c-3691-4418-befc-90080da2feb9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:33.949316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e4cbf-217d-497d-a3e4-bdfb5af5bcb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:33.949982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e4cbf-217d-497d-a3e4-bdfb5af5bcb2 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:33.950265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e4cbf-217d-497d-a3e4-bdfb5af5bcb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:33.950535 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e4cbf-217d-497d-a3e4-bdfb5af5bcb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:33.952219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-aa0e4cbf-217d-497d-a3e4-bdfb5af5bcb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:33.994966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:34.067754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e4cbf-217d-497d-a3e4-bdfb5af5bcb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:34.069597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e4cbf-217d-497d-a3e4-bdfb5af5bcb2 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:34.098129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 683/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:36.546661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7ce61-2371-4934-83f3-56ea5e5328fa None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:36.546661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7ce61-2371-4934-83f3-56ea5e5328fa None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:36.546661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7ce61-2371-4934-83f3-56ea5e5328fa None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:36.546661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 681/1364] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:36.554226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4049f6-2068-4408-979d-0f3f31186895 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:36.554743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4049f6-2068-4408-979d-0f3f31186895 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:36.554864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4049f6-2068-4408-979d-0f3f31186895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:36.645149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3c4049f6-2068-4408-979d-0f3f31186895 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:36.646562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3c4049f6-2068-4408-979d-0f3f31186895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:36.752128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:36.755100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 684/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:15:36.806591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331180ff-0137-4945-b035-407db0fc18fd None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:36.806859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331180ff-0137-4945-b035-407db0fc18fd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:36.807086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331180ff-0137-4945-b035-407db0fc18fd None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:36.809820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 682/1366] 10.222.0.25 () {54 vars in 962 bytes} [Sat Aug 6 01:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:36.815641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193346cf-e350-49a2-a426-251d44e7dc9a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:36.817938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193346cf-e350-49a2-a426-251d44e7dc9a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:36.817938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193346cf-e350-49a2-a426-251d44e7dc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:36.894713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-193346cf-e350-49a2-a426-251d44e7dc9a None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:36.895956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-193346cf-e350-49a2-a426-251d44e7dc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:37.023821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 685/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Sat Aug 6 01:15:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:37.031913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331180ff-0137-4945-b035-407db0fc18fd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:37.033549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-331180ff-0137-4945-b035-407db0fc18fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:37.188980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331180ff-0137-4945-b035-407db0fc18fd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:37.210464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:37.308452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ea5d-4e89-4840-952d-df64c76f2c01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:37.309227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ea5d-4e89-4840-952d-df64c76f2c01 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:37.309626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ea5d-4e89-4840-952d-df64c76f2c01 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:37.309998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ea5d-4e89-4840-952d-df64c76f2c01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:37.311738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-be60ea5d-4e89-4840-952d-df64c76f2c01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:37.403699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ea5d-4e89-4840-952d-df64c76f2c01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:37.405489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ea5d-4e89-4840-952d-df64c76f2c01 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:37.425402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 683/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:38.480258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d8332-368e-4d90-ac6a-d42ea60f746c None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:38.480798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d8332-368e-4d90-ac6a-d42ea60f746c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:38.480798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d8332-368e-4d90-ac6a-d42ea60f746c None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:38.482314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 686/1369] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:38.487297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ae343-cfab-4ef6-b41a-2af274c8ff61 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:38.488361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ae343-cfab-4ef6-b41a-2af274c8ff61 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:38.488470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ae343-cfab-4ef6-b41a-2af274c8ff61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:38.520842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:38.555468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e53ae343-cfab-4ef6-b41a-2af274c8ff61 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:38.556713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e53ae343-cfab-4ef6-b41a-2af274c8ff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:38.660154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 684/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:39.006119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cccd6-5cc0-43ff-b01d-3a8801be8c54 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:39.006119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cccd6-5cc0-43ff-b01d-3a8801be8c54 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:39.006119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cccd6-5cc0-43ff-b01d-3a8801be8c54 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:39.008569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 687/1371] 10.222.0.25 () {54 vars in 964 bytes} [Sat Aug 6 01:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:39.011489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65faa17-861c-4a26-9bd4-360dcc5cf919 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:39.011800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65faa17-861c-4a26-9bd4-360dcc5cf919 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:39.012083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65faa17-861c-4a26-9bd4-360dcc5cf919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:39.094535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d65faa17-861c-4a26-9bd4-360dcc5cf919 None None] MFA Rules not processed for user `ddd0e588d52e40f08a2dc1062ecd4232`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:39.095749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d65faa17-861c-4a26-9bd4-360dcc5cf919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:39.144381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:39.174240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 685/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Sat Aug 6 01:15:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:39.182405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cccd6-5cc0-43ff-b01d-3a8801be8c54 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:39.183948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6d7cccd6-5cc0-43ff-b01d-3a8801be8c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:39.237505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:39.294291 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cccd6-5cc0-43ff-b01d-3a8801be8c54 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:39.371041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb4776-f5c6-4fcb-9042-6e924284134a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:39.371664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb4776-f5c6-4fcb-9042-6e924284134a service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:39.371974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb4776-f5c6-4fcb-9042-6e924284134a service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:39.372264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb4776-f5c6-4fcb-9042-6e924284134a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:39.373857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-28bb4776-f5c6-4fcb-9042-6e924284134a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:39.475487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:39.489968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb4776-f5c6-4fcb-9042-6e924284134a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:39.491441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb4776-f5c6-4fcb-9042-6e924284134a service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:39.511890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 688/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:39.722980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a88d-7100-4cfe-b8bc-abbcf9ce7723 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:39.723646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a88d-7100-4cfe-b8bc-abbcf9ce7723 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:39.723646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a88d-7100-4cfe-b8bc-abbcf9ce7723 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:39.728843 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 686/1374] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:39.730327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ca45a-45e5-446a-93dd-69cab849f5f0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:39.730622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ca45a-45e5-446a-93dd-69cab849f5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:39.730902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ca45a-45e5-446a-93dd-69cab849f5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:39.803669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-686ca45a-45e5-446a-93dd-69cab849f5f0 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:39.804943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-686ca45a-45e5-446a-93dd-69cab849f5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:39.905373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 689/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:39.951209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6a88d-7100-4cfe-b8bc-abbcf9ce7723 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:39.953344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0eb6a88d-7100-4cfe-b8bc-abbcf9ce7723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:40.110356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6a88d-7100-4cfe-b8bc-abbcf9ce7723 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:40.168758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:40.229259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87fe14-a998-4d61-af23-b8e2e805a8a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:40.229881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87fe14-a998-4d61-af23-b8e2e805a8a1 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:40.230164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87fe14-a998-4d61-af23-b8e2e805a8a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:40.230437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87fe14-a998-4d61-af23-b8e2e805a8a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:40.232081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6e87fe14-a998-4d61-af23-b8e2e805a8a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:40.322319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87fe14-a998-4d61-af23-b8e2e805a8a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:40.324099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87fe14-a998-4d61-af23-b8e2e805a8a1 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:40.342243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 687/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:42.473289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122e375-af9e-4d00-9e38-fa474af3cbef None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:42.473822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122e375-af9e-4d00-9e38-fa474af3cbef None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:42.473999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122e375-af9e-4d00-9e38-fa474af3cbef None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:42.475394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 690/1377] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:42.480843 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9935d2-3d19-4ff2-aa52-6fce0a342bce None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:42.481265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9935d2-3d19-4ff2-aa52-6fce0a342bce None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:42.481535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9935d2-3d19-4ff2-aa52-6fce0a342bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:42.564620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9a9935d2-3d19-4ff2-aa52-6fce0a342bce None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:42.566682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9a9935d2-3d19-4ff2-aa52-6fce0a342bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:42.575203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:42.694259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 688/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:42.738763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122e375-af9e-4d00-9e38-fa474af3cbef None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:42.740372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3122e375-af9e-4d00-9e38-fa474af3cbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:42.800898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:42.887722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122e375-af9e-4d00-9e38-fa474af3cbef None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:43.026301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a35a52-eac3-46fb-947e-f7de6c03cc86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:43.027040 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a35a52-eac3-46fb-947e-f7de6c03cc86 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:43.027324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a35a52-eac3-46fb-947e-f7de6c03cc86 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:43.027647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a35a52-eac3-46fb-947e-f7de6c03cc86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:43.029851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e5a35a52-eac3-46fb-947e-f7de6c03cc86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:43.062221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:43.159088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a35a52-eac3-46fb-947e-f7de6c03cc86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:43.164146 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a35a52-eac3-46fb-947e-f7de6c03cc86 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:43.187818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 691/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:45.382814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd22d58-3c31-4d1c-85ca-8aa5ae594a87 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:45.383315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd22d58-3c31-4d1c-85ca-8aa5ae594a87 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:45.383432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd22d58-3c31-4d1c-85ca-8aa5ae594a87 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:45.384917 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 689/1380] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:45.389843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39cb58-5328-4b3a-a64b-53509b61fa6d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:45.390249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39cb58-5328-4b3a-a64b-53509b61fa6d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:45.390435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39cb58-5328-4b3a-a64b-53509b61fa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:45.466807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ce39cb58-5328-4b3a-a64b-53509b61fa6d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:45.468255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ce39cb58-5328-4b3a-a64b-53509b61fa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:45.566798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:45.573667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 692/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:45.617605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd22d58-3c31-4d1c-85ca-8aa5ae594a87 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:45.619202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6bd22d58-3c31-4d1c-85ca-8aa5ae594a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:45.761564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:45.773069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd22d58-3c31-4d1c-85ca-8aa5ae594a87 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:45.894391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ac475-3700-4e2d-82ae-5c6c5f26fcac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:45.894948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ac475-3700-4e2d-82ae-5c6c5f26fcac service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:45.895225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ac475-3700-4e2d-82ae-5c6c5f26fcac service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:45.895497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ac475-3700-4e2d-82ae-5c6c5f26fcac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:45.897548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5e6ac475-3700-4e2d-82ae-5c6c5f26fcac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:45.999210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ac475-3700-4e2d-82ae-5c6c5f26fcac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:46.000857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ac475-3700-4e2d-82ae-5c6c5f26fcac service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:46.011733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:46.020658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 690/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:48.582901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571915fb-416c-4a51-8936-3e29ba8d549f None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:48.583408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571915fb-416c-4a51-8936-3e29ba8d549f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:48.583537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571915fb-416c-4a51-8936-3e29ba8d549f None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:48.584970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 693/1383] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:48.594832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb64a15-e15a-438c-8e7d-c6119cc85ef7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:48.595389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb64a15-e15a-438c-8e7d-c6119cc85ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:48.595897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb64a15-e15a-438c-8e7d-c6119cc85ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:48.688542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4bb64a15-e15a-438c-8e7d-c6119cc85ef7 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:48.689833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4bb64a15-e15a-438c-8e7d-c6119cc85ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:48.801724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 691/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:48.847696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571915fb-416c-4a51-8936-3e29ba8d549f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:48.849237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-571915fb-416c-4a51-8936-3e29ba8d549f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:48.999871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571915fb-416c-4a51-8936-3e29ba8d549f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:49.083236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:49.130722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb47e64-ed6f-4565-8184-0bcf803a9c1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:49.131375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb47e64-ed6f-4565-8184-0bcf803a9c1b service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:49.131673 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb47e64-ed6f-4565-8184-0bcf803a9c1b service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:49.131992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb47e64-ed6f-4565-8184-0bcf803a9c1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:49.133713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ecb47e64-ed6f-4565-8184-0bcf803a9c1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:49.223599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb47e64-ed6f-4565-8184-0bcf803a9c1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:49.225330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb47e64-ed6f-4565-8184-0bcf803a9c1b service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:49.244708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 694/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:15:51.328369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857ece-175b-43ee-8533-1f3e23fe6a8c None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:51.328881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857ece-175b-43ee-8533-1f3e23fe6a8c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:51.328881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857ece-175b-43ee-8533-1f3e23fe6a8c None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:51.330290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 692/1386] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:51.336428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559bd84-ba7b-409f-8e05-d35a4ec54f20 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:51.336964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559bd84-ba7b-409f-8e05-d35a4ec54f20 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:51.336964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559bd84-ba7b-409f-8e05-d35a4ec54f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:51.404294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e559bd84-ba7b-409f-8e05-d35a4ec54f20 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:51.405792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e559bd84-ba7b-409f-8e05-d35a4ec54f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:51.445302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:51.513860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 695/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:51.567411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857ece-175b-43ee-8533-1f3e23fe6a8c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:51.569222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d5857ece-175b-43ee-8533-1f3e23fe6a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:51.657472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:51.729646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857ece-175b-43ee-8533-1f3e23fe6a8c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:51.844615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e41d4a-bddc-44d5-a2b8-c44f2bee8f39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:51.845415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e41d4a-bddc-44d5-a2b8-c44f2bee8f39 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:51.845799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e41d4a-bddc-44d5-a2b8-c44f2bee8f39 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:51.846416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e41d4a-bddc-44d5-a2b8-c44f2bee8f39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:51.849128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-07e41d4a-bddc-44d5-a2b8-c44f2bee8f39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:51.894691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:51.949823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e41d4a-bddc-44d5-a2b8-c44f2bee8f39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:51.951526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e41d4a-bddc-44d5-a2b8-c44f2bee8f39 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:51.971049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 693/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:54.141469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f1096-9bcd-499e-a92d-0deb58bacd6a None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:54.142013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f1096-9bcd-499e-a92d-0deb58bacd6a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:54.142013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f1096-9bcd-499e-a92d-0deb58bacd6a None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:54.143307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 696/1389] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:54.149279 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c6e17-f900-4c33-8b0e-2811db3546e2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:54.149722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c6e17-f900-4c33-8b0e-2811db3546e2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:54.149930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c6e17-f900-4c33-8b0e-2811db3546e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:54.216905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d60c6e17-f900-4c33-8b0e-2811db3546e2 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:54.218099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d60c6e17-f900-4c33-8b0e-2811db3546e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:54.314893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 694/1390] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:54.359125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f1096-9bcd-499e-a92d-0deb58bacd6a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:54.360773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b79f1096-9bcd-499e-a92d-0deb58bacd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:54.506285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f1096-9bcd-499e-a92d-0deb58bacd6a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:54.523643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:54.625209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b21a0-e056-4949-864a-77491efbfd10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:54.625835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b21a0-e056-4949-864a-77491efbfd10 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:54.626118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b21a0-e056-4949-864a-77491efbfd10 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:54.626395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b21a0-e056-4949-864a-77491efbfd10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:54.628033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ad0b21a0-e056-4949-864a-77491efbfd10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:54.716806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b21a0-e056-4949-864a-77491efbfd10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:54.718426 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b21a0-e056-4949-864a-77491efbfd10 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:54.736712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 697/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:54.987395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6db52-466c-4e87-8acc-4bcaec439dd3 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:54.987674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6db52-466c-4e87-8acc-4bcaec439dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:54.987993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6db52-466c-4e87-8acc-4bcaec439dd3 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:54.989368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 695/1392] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:54.995380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2f3b6-2e3b-429d-8303-f3ed0a95a6b8 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:54.995670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2f3b6-2e3b-429d-8303-f3ed0a95a6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:54.995986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2f3b6-2e3b-429d-8303-f3ed0a95a6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:55.024310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:55.076246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e2f2f3b6-2e3b-429d-8303-f3ed0a95a6b8 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:55.077389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e2f2f3b6-2e3b-429d-8303-f3ed0a95a6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:55.183045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 698/1393] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:55.239293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b88ec6-a33a-4e43-bf55-9d426b047870 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:55.239731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b88ec6-a33a-4e43-bf55-9d426b047870 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:55.239894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b88ec6-a33a-4e43-bf55-9d426b047870 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:55.241405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 696/1394] 10.222.0.25 () {54 vars in 966 bytes} [Sat Aug 6 01:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:55.246169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dde91-2806-4de8-823e-6478765d420e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:55.246486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dde91-2806-4de8-823e-6478765d420e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:55.246802 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dde91-2806-4de8-823e-6478765d420e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:55.321689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3a3dde91-2806-4de8-823e-6478765d420e None None] MFA Rules not processed for user `9a234a96eccd4f38b0213d7d0461ad24`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:55.322902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3a3dde91-2806-4de8-823e-6478765d420e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:55.365388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:55.404953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 699/1395] 10.222.0.25 () {58 vars in 1063 bytes} [Sat Aug 6 01:15:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:55.412742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b88ec6-a33a-4e43-bf55-9d426b047870 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:55.414245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e4b88ec6-a33a-4e43-bf55-9d426b047870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:55.428090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:55.517887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b88ec6-a33a-4e43-bf55-9d426b047870 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:55.590217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a376ace-211a-458f-892f-ac826ac9f03c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a234a96eccd4f38b0213d7d0461ad24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:55.591132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a376ace-211a-458f-892f-ac826ac9f03c service glance] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:55.591420 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a376ace-211a-458f-892f-ac826ac9f03c service glance] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:55.591693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a376ace-211a-458f-892f-ac826ac9f03c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:55.593345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1a376ace-211a-458f-892f-ac826ac9f03c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:55.660966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:55.701993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a376ace-211a-458f-892f-ac826ac9f03c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:55.703426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a376ace-211a-458f-892f-ac826ac9f03c service glance] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:55.721803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 697/1396] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:15:56.568303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3cf5b-a0dc-419e-9c27-a425562b518a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:56.568774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3cf5b-a0dc-419e-9c27-a425562b518a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:56.568872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3cf5b-a0dc-419e-9c27-a425562b518a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:56.641030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-30d3cf5b-a0dc-419e-9c27-a425562b518a None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:56.642254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-30d3cf5b-a0dc-419e-9c27-a425562b518a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:56.728538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 700/1397] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:15:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:56.771582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31761b-aa1c-40e8-97d3-e20e8b2f6f24 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:56.772115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31761b-aa1c-40e8-97d3-e20e8b2f6f24 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:56.773813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31761b-aa1c-40e8-97d3-e20e8b2f6f24 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:56.775261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 698/1398] 10.222.0.25 () {54 vars in 943 bytes} [Sat Aug 6 01:15:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:56.782385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c02ab1-1c85-48ec-83b7-58034a03fae0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:56.782672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c02ab1-1c85-48ec-83b7-58034a03fae0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:56.782948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c02ab1-1c85-48ec-83b7-58034a03fae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:56.851512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:56.858596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-96c02ab1-1c85-48ec-83b7-58034a03fae0 None None] MFA Rules not processed for user `33aa626411f1474595102a4541af901a`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:56.859813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-96c02ab1-1c85-48ec-83b7-58034a03fae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:56.882487 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63990aa-ceed-49e4-a8bf-cd380c615656 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:56.882857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63990aa-ceed-49e4-a8bf-cd380c615656 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:56.883134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63990aa-ceed-49e4-a8bf-cd380c615656 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:56.884508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 699/1399] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:15:56.889391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c3eb-e5df-4704-8603-53f4df943dc6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:56.889754 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c3eb-e5df-4704-8603-53f4df943dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:56.890039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c3eb-e5df-4704-8603-53f4df943dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:56.957360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-90b5c3eb-e5df-4704-8603-53f4df943dc6 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:56.958576 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-90b5c3eb-e5df-4704-8603-53f4df943dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:56.974799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 701/1400] 10.222.0.25 () {58 vars in 1040 bytes} [Sat Aug 6 01:15:56 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:56.985280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c02ab1-1c85-48ec-83b7-58034a03fae0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:56.995728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c02ab1-1c85-48ec-83b7-58034a03fae0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:57.078119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 700/1401] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:57.120815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e2834-39e9-4774-a13a-1ece7d31af5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:57.122305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e2834-39e9-4774-a13a-1ece7d31af5c service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:57.122581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e2834-39e9-4774-a13a-1ece7d31af5c service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:57.122846 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e2834-39e9-4774-a13a-1ece7d31af5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:57.122972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5c3eb-e5df-4704-8603-53f4df943dc6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:57.124206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-90b5c3eb-e5df-4704-8603-53f4df943dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:57.124505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-679e2834-39e9-4774-a13a-1ece7d31af5c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:57.127163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:57.152299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:57.209747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e2834-39e9-4774-a13a-1ece7d31af5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:57.211342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e2834-39e9-4774-a13a-1ece7d31af5c service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:57.219708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 702/1402] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:15:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 01:15:57.241197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5c3eb-e5df-4704-8603-53f4df943dc6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:15:57.354577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20628e-fe03-429b-b623-aab23795374d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:15:57.355190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20628e-fe03-429b-b623-aab23795374d service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:57.355470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20628e-fe03-429b-b623-aab23795374d service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:57.355746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20628e-fe03-429b-b623-aab23795374d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:57.357401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5c20628e-fe03-429b-b623-aab23795374d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:57.375718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:57.445551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20628e-fe03-429b-b623-aab23795374d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:15:57.447170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20628e-fe03-429b-b623-aab23795374d service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:15:57.466022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 701/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:15:59.662338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d5d1d-e970-4b18-91c7-60733a600e84 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:59.662898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d5d1d-e970-4b18-91c7-60733a600e84 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:59.662898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d5d1d-e970-4b18-91c7-60733a600e84 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:59.664594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 703/1404] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:15:59.670314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eddd7f-660f-4474-b344-f1d40c8990e6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:15:59.670850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eddd7f-660f-4474-b344-f1d40c8990e6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:15:59.670978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eddd7f-660f-4474-b344-f1d40c8990e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:15:59.749574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c3eddd7f-660f-4474-b344-f1d40c8990e6 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:15:59.750783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c3eddd7f-660f-4474-b344-f1d40c8990e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:15:59.854843 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:15:59.861082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 702/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:15:59.906002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d5d1d-e970-4b18-91c7-60733a600e84 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:15:59.907531 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5f3d5d1d-e970-4b18-91c7-60733a600e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:00.194463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:00.197119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d5d1d-e970-4b18-91c7-60733a600e84 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:00.323709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63f779-0fb9-4ef5-b316-8fd614e70b91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:00.324517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63f779-0fb9-4ef5-b316-8fd614e70b91 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:00.324808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63f779-0fb9-4ef5-b316-8fd614e70b91 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:00.325085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63f779-0fb9-4ef5-b316-8fd614e70b91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:00.326712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ed63f779-0fb9-4ef5-b316-8fd614e70b91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:00.432029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63f779-0fb9-4ef5-b316-8fd614e70b91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:00.433655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63f779-0fb9-4ef5-b316-8fd614e70b91 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:00.455122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 704/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:16:02.688520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b731af-2e5f-4d57-82f9-8023db22bc33 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:02.689060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b731af-2e5f-4d57-82f9-8023db22bc33 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:02.689157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b731af-2e5f-4d57-82f9-8023db22bc33 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:02.693588 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 703/1407] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:16:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:16:02.699323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914b8f4-f3a6-4616-a654-cb9b696aba3d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:02.699949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914b8f4-f3a6-4616-a654-cb9b696aba3d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:02.699949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914b8f4-f3a6-4616-a654-cb9b696aba3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:02.705012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:02.785527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d914b8f4-f3a6-4616-a654-cb9b696aba3d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:02.786742 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d914b8f4-f3a6-4616-a654-cb9b696aba3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:02.892847 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 705/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:02.941663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b731af-2e5f-4d57-82f9-8023db22bc33 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:02.942404 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f4b731af-2e5f-4d57-82f9-8023db22bc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:03.120894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b731af-2e5f-4d57-82f9-8023db22bc33 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:03.195172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:03.244093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c2dd2-24da-486d-8b5d-a9595b946119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:03.244709 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c2dd2-24da-486d-8b5d-a9595b946119 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:03.244987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c2dd2-24da-486d-8b5d-a9595b946119 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:03.245267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c2dd2-24da-486d-8b5d-a9595b946119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:03.247706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a64c2dd2-24da-486d-8b5d-a9595b946119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:03.351609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c2dd2-24da-486d-8b5d-a9595b946119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:03.353339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c2dd2-24da-486d-8b5d-a9595b946119 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:03.374956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 704/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:13.161271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=64 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:13.372260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=64 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:20.704198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baddc33-7a31-429a-a072-e6061a25b7c4 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:20.704760 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baddc33-7a31-429a-a072-e6061a25b7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:20.704760 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baddc33-7a31-429a-a072-e6061a25b7c4 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:20.706670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 706/1410] 10.222.0.25 () {54 vars in 918 bytes} [Sat Aug 6 01:16:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:20.711758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328f51d-44fb-4437-810a-a4394c8080e5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:20.712188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328f51d-44fb-4437-810a-a4394c8080e5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:20.712351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328f51d-44fb-4437-810a-a4394c8080e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:20.787258 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a328f51d-44fb-4437-810a-a4394c8080e5 None None] MFA Rules not processed for user `8361f2f9c70b4a3288a187cb2d97d415`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:20.788489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a328f51d-44fb-4437-810a-a4394c8080e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:20.892867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 705/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Aug 6 01:16:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:20.907017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e299a-922f-4843-b8d0-eaf541c9a014 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:20.907303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e299a-922f-4843-b8d0-eaf541c9a014 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:20.907578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e299a-922f-4843-b8d0-eaf541c9a014 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:20.908945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 707/1412] 10.222.0.25 () {54 vars in 918 bytes} [Sat Aug 6 01:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:20.913772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb2ac0-228f-4705-96ff-4ac74da9d4db None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:20.914062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb2ac0-228f-4705-96ff-4ac74da9d4db None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:20.914340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb2ac0-228f-4705-96ff-4ac74da9d4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:20.989153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:20.992111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-dadb2ac0-228f-4705-96ff-4ac74da9d4db None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:20.993332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-dadb2ac0-228f-4705-96ff-4ac74da9d4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:21.111027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 706/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Aug 6 01:16:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:23.570897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be684b-b885-48be-9218-ca4a2d2cf684 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:23.571438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be684b-b885-48be-9218-ca4a2d2cf684 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:23.571438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be684b-b885-48be-9218-ca4a2d2cf684 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:23.572805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:23.577943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81cf7d-f7e3-4677-baa3-90df87ab6d70 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:23.578398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81cf7d-f7e3-4677-baa3-90df87ab6d70 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:23.578495 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81cf7d-f7e3-4677-baa3-90df87ab6d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:23.643384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8d81cf7d-f7e3-4677-baa3-90df87ab6d70 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:23.645146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8d81cf7d-f7e3-4677-baa3-90df87ab6d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:23.698152 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:23.740468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:23.782509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be684b-b885-48be-9218-ca4a2d2cf684 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:23.784065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c6be684b-b885-48be-9218-ca4a2d2cf684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:23.923891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be684b-b885-48be-9218-ca4a2d2cf684 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:23.994090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:24.033930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c87e7-a5fb-4d54-9782-15e868550e3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:24.035068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c87e7-a5fb-4d54-9782-15e868550e3c service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:24.035345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c87e7-a5fb-4d54-9782-15e868550e3c service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:24.035615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c87e7-a5fb-4d54-9782-15e868550e3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:24.037283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f59c87e7-a5fb-4d54-9782-15e868550e3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:24.123108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c87e7-a5fb-4d54-9782-15e868550e3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:24.124650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c87e7-a5fb-4d54-9782-15e868550e3c service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:24.142760 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 709/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:25.031514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163ebc2-e0aa-40e4-8514-ac3cab8d5656 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:25.032105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163ebc2-e0aa-40e4-8514-ac3cab8d5656 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:25.032105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163ebc2-e0aa-40e4-8514-ac3cab8d5656 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:25.033459 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:25.038898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a786efe-03b2-4d92-9a70-d128d74cc720 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:25.039795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a786efe-03b2-4d92-9a70-d128d74cc720 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:25.040283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a786efe-03b2-4d92-9a70-d128d74cc720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:25.114215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6a786efe-03b2-4d92-9a70-d128d74cc720 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:25.116101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6a786efe-03b2-4d92-9a70-d128d74cc720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:25.142369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:25.210940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:25.366607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163ebc2-e0aa-40e4-8514-ac3cab8d5656 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:25.368066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6163ebc2-e0aa-40e4-8514-ac3cab8d5656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:25.509081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163ebc2-e0aa-40e4-8514-ac3cab8d5656 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:25.548397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:25.740402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116e559-860b-4e72-b818-15d2ff82ff3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:25.741018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116e559-860b-4e72-b818-15d2ff82ff3d service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:25.741299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116e559-860b-4e72-b818-15d2ff82ff3d service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:25.741573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116e559-860b-4e72-b818-15d2ff82ff3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:25.743221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d116e559-860b-4e72-b818-15d2ff82ff3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:25.833251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116e559-860b-4e72-b818-15d2ff82ff3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:25.834830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116e559-860b-4e72-b818-15d2ff82ff3d service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:25.852929 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 709/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:26.754487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d4abe-a2c7-44cc-8b1a-2d02ac9ec78a None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:26.755013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d4abe-a2c7-44cc-8b1a-2d02ac9ec78a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:26.755013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d4abe-a2c7-44cc-8b1a-2d02ac9ec78a None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:26.756405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:26.761269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994c1c8-780b-46f2-be10-b4e44c044d45 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:26.761653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994c1c8-780b-46f2-be10-b4e44c044d45 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:26.761826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994c1c8-780b-46f2-be10-b4e44c044d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:26.816788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:26.838057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e994c1c8-780b-46f2-be10-b4e44c044d45 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:26.839203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e994c1c8-780b-46f2-be10-b4e44c044d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:26.933882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:27.087090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d4abe-a2c7-44cc-8b1a-2d02ac9ec78a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:27.088722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e96d4abe-a2c7-44cc-8b1a-2d02ac9ec78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:27.236119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d4abe-a2c7-44cc-8b1a-2d02ac9ec78a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:27.248398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:27.353440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdc6c2-7ee2-465f-a4b3-f278ac7fca72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:27.353973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdc6c2-7ee2-465f-a4b3-f278ac7fca72 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:27.354254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdc6c2-7ee2-465f-a4b3-f278ac7fca72 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:27.354526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdc6c2-7ee2-465f-a4b3-f278ac7fca72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:27.356169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e4bdc6c2-7ee2-465f-a4b3-f278ac7fca72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:27.444353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdc6c2-7ee2-465f-a4b3-f278ac7fca72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:27.449479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdc6c2-7ee2-465f-a4b3-f278ac7fca72 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:27.464072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 712/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:28.322982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2d831-47b6-486b-9e39-563e2ca64d99 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:28.323540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2d831-47b6-486b-9e39-563e2ca64d99 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:28.323540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2d831-47b6-486b-9e39-563e2ca64d99 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:28.324951 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:28.330117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f994e5d-c70f-42a4-9ebc-07eef8b623ec None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:28.330519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f994e5d-c70f-42a4-9ebc-07eef8b623ec None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:28.330682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f994e5d-c70f-42a4-9ebc-07eef8b623ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:28.354686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:28.409494 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8f994e5d-c70f-42a4-9ebc-07eef8b623ec None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:28.410701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8f994e5d-c70f-42a4-9ebc-07eef8b623ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:28.509119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:28.659922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2d831-47b6-486b-9e39-563e2ca64d99 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:28.661483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-53a2d831-47b6-486b-9e39-563e2ca64d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:28.769293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:28.805694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2d831-47b6-486b-9e39-563e2ca64d99 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:28.916445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524841c-b5d8-4f1f-b6bf-411623ef4ea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:28.916989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524841c-b5d8-4f1f-b6bf-411623ef4ea8 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:28.917269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524841c-b5d8-4f1f-b6bf-411623ef4ea8 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:28.917541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524841c-b5d8-4f1f-b6bf-411623ef4ea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:28.919139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1524841c-b5d8-4f1f-b6bf-411623ef4ea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:28.993555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:29.007038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524841c-b5d8-4f1f-b6bf-411623ef4ea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:29.008641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524841c-b5d8-4f1f-b6bf-411623ef4ea8 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:29.027079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 712/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:29.885890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b88b6-ea54-40b9-bf5e-c59585b46004 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:29.886318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b88b6-ea54-40b9-bf5e-c59585b46004 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:29.886436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b88b6-ea54-40b9-bf5e-c59585b46004 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:29.887825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:29.892696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec898ce2-7680-49ea-a45b-938ffda1e9c7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:29.893125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec898ce2-7680-49ea-a45b-938ffda1e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:29.893227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec898ce2-7680-49ea-a45b-938ffda1e9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:29.966370 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ec898ce2-7680-49ea-a45b-938ffda1e9c7 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:29.967836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ec898ce2-7680-49ea-a45b-938ffda1e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:30.064516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:30.214190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b88b6-ea54-40b9-bf5e-c59585b46004 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:30.215694 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-283b88b6-ea54-40b9-bf5e-c59585b46004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:30.297039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:30.387610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b88b6-ea54-40b9-bf5e-c59585b46004 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:30.498289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf66f81-8352-4eb6-8c20-0c2a1f46524a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:30.499209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf66f81-8352-4eb6-8c20-0c2a1f46524a service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:30.499708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf66f81-8352-4eb6-8c20-0c2a1f46524a service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:30.500243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf66f81-8352-4eb6-8c20-0c2a1f46524a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:30.503622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2bf66f81-8352-4eb6-8c20-0c2a1f46524a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:30.551441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:30.592603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf66f81-8352-4eb6-8c20-0c2a1f46524a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:30.594243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf66f81-8352-4eb6-8c20-0c2a1f46524a service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:30.611992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 715/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:31.480785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a276-aa41-485a-8ce5-aa540aec6a72 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:31.481336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a276-aa41-485a-8ce5-aa540aec6a72 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:31.481336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a276-aa41-485a-8ce5-aa540aec6a72 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:31.482686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:31.488325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a775c15-4887-4c01-869e-b9ad85345294 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:31.488751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a775c15-4887-4c01-869e-b9ad85345294 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:31.488862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a775c15-4887-4c01-869e-b9ad85345294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:31.572478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1a775c15-4887-4c01-869e-b9ad85345294 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:31.573743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1a775c15-4887-4c01-869e-b9ad85345294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:31.681744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:31.834758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509a276-aa41-485a-8ce5-aa540aec6a72 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:31.837413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1509a276-aa41-485a-8ce5-aa540aec6a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:31.887101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:31.984923 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509a276-aa41-485a-8ce5-aa540aec6a72 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:32.099080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6bb63-88be-4d0b-b187-334e1d0e5376 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:32.099638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6bb63-88be-4d0b-b187-334e1d0e5376 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:32.099914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6bb63-88be-4d0b-b187-334e1d0e5376 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:32.100188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6bb63-88be-4d0b-b187-334e1d0e5376 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:32.101797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ffc6bb63-88be-4d0b-b187-334e1d0e5376 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:32.108802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:32.193986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6bb63-88be-4d0b-b187-334e1d0e5376 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:32.195568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6bb63-88be-4d0b-b187-334e1d0e5376 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:32.214690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 715/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:33.130067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a9a69-b186-4135-91b8-d222b9f7f69e None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:33.130613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a9a69-b186-4135-91b8-d222b9f7f69e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:33.130613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a9a69-b186-4135-91b8-d222b9f7f69e None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:33.131921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:33.136893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b8601-5de9-496b-a664-b466d4e79d31 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:33.137333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b8601-5de9-496b-a664-b466d4e79d31 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:33.137433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b8601-5de9-496b-a664-b466d4e79d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:33.217739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-dc5b8601-5de9-496b-a664-b466d4e79d31 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:33.218933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-dc5b8601-5de9-496b-a664-b466d4e79d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:33.283788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:33.311168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:33.461721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a9a69-b186-4135-91b8-d222b9f7f69e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:33.463295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3d3a9a69-b186-4135-91b8-d222b9f7f69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:33.482650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:33.608784 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a9a69-b186-4135-91b8-d222b9f7f69e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:33.693148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:33.719221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44441db8-e490-49b5-a36d-eda342d1a5e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:33.719800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44441db8-e490-49b5-a36d-eda342d1a5e7 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:33.720078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44441db8-e490-49b5-a36d-eda342d1a5e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:33.720347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44441db8-e490-49b5-a36d-eda342d1a5e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:33.723145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-44441db8-e490-49b5-a36d-eda342d1a5e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:33.808632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44441db8-e490-49b5-a36d-eda342d1a5e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:33.810209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44441db8-e490-49b5-a36d-eda342d1a5e7 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:33.827516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 718/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:34.688406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734c36a-a11e-4690-bb78-a430c9f8841b None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:34.688936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734c36a-a11e-4690-bb78-a430c9f8841b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:34.688936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734c36a-a11e-4690-bb78-a430c9f8841b None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:34.690290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:34.695874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5465eb-9b3a-4504-857d-c5b72eaa48bc None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:34.696311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5465eb-9b3a-4504-857d-c5b72eaa48bc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:34.696417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5465eb-9b3a-4504-857d-c5b72eaa48bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:34.775501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3b5465eb-9b3a-4504-857d-c5b72eaa48bc None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:34.776747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3b5465eb-9b3a-4504-857d-c5b72eaa48bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:34.818669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:34.873060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:35.022060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734c36a-a11e-4690-bb78-a430c9f8841b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:35.023578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3734c36a-a11e-4690-bb78-a430c9f8841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:35.170229 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734c36a-a11e-4690-bb78-a430c9f8841b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:35.224466 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:35.281266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad230847-ca4f-475c-87ca-9c94c8cdfa7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:35.281883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad230847-ca4f-475c-87ca-9c94c8cdfa7e service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:35.282163 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad230847-ca4f-475c-87ca-9c94c8cdfa7e service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:35.282440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad230847-ca4f-475c-87ca-9c94c8cdfa7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:35.284072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ad230847-ca4f-475c-87ca-9c94c8cdfa7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:35.374060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad230847-ca4f-475c-87ca-9c94c8cdfa7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:35.375643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad230847-ca4f-475c-87ca-9c94c8cdfa7e service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:35.394269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 718/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:36.252323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5092170-a429-4e63-b607-7a9d61389932 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:36.253407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5092170-a429-4e63-b607-7a9d61389932 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:36.253531 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5092170-a429-4e63-b607-7a9d61389932 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:36.254912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:36.259766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11822e9c-f700-448f-9ac5-2bbb6043d1b1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:36.260213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11822e9c-f700-448f-9ac5-2bbb6043d1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:36.260341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11822e9c-f700-448f-9ac5-2bbb6043d1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:36.336212 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-11822e9c-f700-448f-9ac5-2bbb6043d1b1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:36.337411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-11822e9c-f700-448f-9ac5-2bbb6043d1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:36.339379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:36.431343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:36.585180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5092170-a429-4e63-b607-7a9d61389932 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:36.586683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c5092170-a429-4e63-b607-7a9d61389932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:36.728279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5092170-a429-4e63-b607-7a9d61389932 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:36.754323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:36.835471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dbca1-4667-4e5c-bfaf-4459f0e2f657 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:36.836051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dbca1-4667-4e5c-bfaf-4459f0e2f657 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:36.836334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dbca1-4667-4e5c-bfaf-4459f0e2f657 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:36.836632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dbca1-4667-4e5c-bfaf-4459f0e2f657 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:36.838284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-444dbca1-4667-4e5c-bfaf-4459f0e2f657 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:36.928381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dbca1-4667-4e5c-bfaf-4459f0e2f657 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:36.933265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dbca1-4667-4e5c-bfaf-4459f0e2f657 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:36.947902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 721/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:38.840908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc8751-12a1-4048-9ee5-77efc75eb813 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:38.841442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc8751-12a1-4048-9ee5-77efc75eb813 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:38.841442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc8751-12a1-4048-9ee5-77efc75eb813 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:38.843094 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:38.849227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cb63f-fb58-4e22-8ecd-f3ce87ef806c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:38.849639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cb63f-fb58-4e22-8ecd-f3ce87ef806c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:38.849790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cb63f-fb58-4e22-8ecd-f3ce87ef806c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:38.890128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:38.914995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b10cb63f-fb58-4e22-8ecd-f3ce87ef806c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:38.916218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b10cb63f-fb58-4e22-8ecd-f3ce87ef806c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:39.011433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:39.050564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc8751-12a1-4048-9ee5-77efc75eb813 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:39.052119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c9cc8751-12a1-4048-9ee5-77efc75eb813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:39.153162 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc8751-12a1-4048-9ee5-77efc75eb813 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:39.189070 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:39.230118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00774d05-5e8c-4743-90b9-3f8de2b00a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:39.230734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774d05-5e8c-4743-90b9-3f8de2b00a59 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:39.231014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774d05-5e8c-4743-90b9-3f8de2b00a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:39.231283 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774d05-5e8c-4743-90b9-3f8de2b00a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:39.232939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-00774d05-5e8c-4743-90b9-3f8de2b00a59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:39.345417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00774d05-5e8c-4743-90b9-3f8de2b00a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:39.346883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00774d05-5e8c-4743-90b9-3f8de2b00a59 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:39.365695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 721/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:41.537922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5bb5e-586f-4de7-951c-fbfe488cffab None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:41.538518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5bb5e-586f-4de7-951c-fbfe488cffab None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:41.538518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5bb5e-586f-4de7-951c-fbfe488cffab None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:41.539834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:41.545072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d4dab-0e92-4ff1-9994-7f7854015c8b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:41.545504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d4dab-0e92-4ff1-9994-7f7854015c8b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:41.545635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d4dab-0e92-4ff1-9994-7f7854015c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:41.610845 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-539d4dab-0e92-4ff1-9994-7f7854015c8b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:41.612091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-539d4dab-0e92-4ff1-9994-7f7854015c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:41.625334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:41.707462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:41.745475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5bb5e-586f-4de7-951c-fbfe488cffab None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:41.747033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d7f5bb5e-586f-4de7-951c-fbfe488cffab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:41.847610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5bb5e-586f-4de7-951c-fbfe488cffab None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:41.859373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:41.919267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e963b-58c3-457b-bc5d-9a8e0b5cfb61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:41.919921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e963b-58c3-457b-bc5d-9a8e0b5cfb61 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:41.920202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e963b-58c3-457b-bc5d-9a8e0b5cfb61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:41.920474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e963b-58c3-457b-bc5d-9a8e0b5cfb61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:41.922079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1a2e963b-58c3-457b-bc5d-9a8e0b5cfb61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:42.027080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e963b-58c3-457b-bc5d-9a8e0b5cfb61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:42.028553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e963b-58c3-457b-bc5d-9a8e0b5cfb61 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:42.045915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 724/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:44.404099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a1e1d-b1bc-47e3-9567-9f28689e5849 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:44.404637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a1e1d-b1bc-47e3-9567-9f28689e5849 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:44.404637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a1e1d-b1bc-47e3-9567-9f28689e5849 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:44.405997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:44.411710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9057b-8b02-40cd-8b7e-80f7204753e4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:44.412167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9057b-8b02-40cd-8b7e-80f7204753e4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:44.412279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9057b-8b02-40cd-8b7e-80f7204753e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:44.458896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:44.476583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-27a9057b-8b02-40cd-8b7e-80f7204753e4 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:44.477792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-27a9057b-8b02-40cd-8b7e-80f7204753e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:44.571545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:44.611645 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a1e1d-b1bc-47e3-9567-9f28689e5849 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:44.613703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d27a1e1d-b1bc-47e3-9567-9f28689e5849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:44.716764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a1e1d-b1bc-47e3-9567-9f28689e5849 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:44.759736 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:44.794361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322cec6-b99a-4012-94e4-c7e8866242d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:44.794977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322cec6-b99a-4012-94e4-c7e8866242d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:44.795258 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322cec6-b99a-4012-94e4-c7e8866242d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:44.795532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322cec6-b99a-4012-94e4-c7e8866242d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:44.797175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1322cec6-b99a-4012-94e4-c7e8866242d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:44.903320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322cec6-b99a-4012-94e4-c7e8866242d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:44.904781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322cec6-b99a-4012-94e4-c7e8866242d0 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:44.922794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 724/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:47.121729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166bd66-30c6-467b-abd1-65644cf77364 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:47.122290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166bd66-30c6-467b-abd1-65644cf77364 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:47.122290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166bd66-30c6-467b-abd1-65644cf77364 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:47.123692 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:47.129575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155adff8-4e5b-44ee-b06a-976c48b44ca3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:47.129998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155adff8-4e5b-44ee-b06a-976c48b44ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:47.130125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155adff8-4e5b-44ee-b06a-976c48b44ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:47.195674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-155adff8-4e5b-44ee-b06a-976c48b44ca3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:47.196890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-155adff8-4e5b-44ee-b06a-976c48b44ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:47.217300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:47.292653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:16:47.332069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166bd66-30c6-467b-abd1-65644cf77364 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:47.333584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9166bd66-30c6-467b-abd1-65644cf77364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:47.441375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166bd66-30c6-467b-abd1-65644cf77364 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:47.461037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:47.514459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc6270-260f-4cd0-b614-e2c2e2f630c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:47.515076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc6270-260f-4cd0-b614-e2c2e2f630c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:47.515383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc6270-260f-4cd0-b614-e2c2e2f630c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:47.515654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc6270-260f-4cd0-b614-e2c2e2f630c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:47.517291 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2afc6270-260f-4cd0-b614-e2c2e2f630c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:47.623444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc6270-260f-4cd0-b614-e2c2e2f630c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:47.624955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc6270-260f-4cd0-b614-e2c2e2f630c2 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:47.642634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 727/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:49.685688 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934a04-b8f2-406b-8bea-898a71b9fa3e None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:49.686224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934a04-b8f2-406b-8bea-898a71b9fa3e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:49.686224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46934a04-b8f2-406b-8bea-898a71b9fa3e None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:49.687609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:49.693385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59e3a0-5057-47b3-82bc-39eecf5c3bbb None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:49.693808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59e3a0-5057-47b3-82bc-39eecf5c3bbb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:49.693948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59e3a0-5057-47b3-82bc-39eecf5c3bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:49.756400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:49.758886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-fd59e3a0-5057-47b3-82bc-39eecf5c3bbb None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:49.760131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fd59e3a0-5057-47b3-82bc-39eecf5c3bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:49.854597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:49.893309 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46934a04-b8f2-406b-8bea-898a71b9fa3e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:49.894837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-46934a04-b8f2-406b-8bea-898a71b9fa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:49.995689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46934a04-b8f2-406b-8bea-898a71b9fa3e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:50.042878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:50.072279 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4829f-74a4-4356-b4f1-6bfca1dec2ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:50.072895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4829f-74a4-4356-b4f1-6bfca1dec2ce service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:50.073175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4829f-74a4-4356-b4f1-6bfca1dec2ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:50.073446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4829f-74a4-4356-b4f1-6bfca1dec2ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:50.075050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d6a4829f-74a4-4356-b4f1-6bfca1dec2ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:50.184301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4829f-74a4-4356-b4f1-6bfca1dec2ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:50.185651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4829f-74a4-4356-b4f1-6bfca1dec2ce service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:50.203069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 727/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:52.323391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fb9c0-ba9d-45a7-b6a7-49d7990d6029 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:52.323989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fb9c0-ba9d-45a7-b6a7-49d7990d6029 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:52.323989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fb9c0-ba9d-45a7-b6a7-49d7990d6029 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:52.325406 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:52.330751 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cd269-3c6e-4671-94b5-7506a56dfb4c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:52.331670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cd269-3c6e-4671-94b5-7506a56dfb4c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:52.331895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cd269-3c6e-4671-94b5-7506a56dfb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:52.399251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b13cd269-3c6e-4671-94b5-7506a56dfb4c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:52.400556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b13cd269-3c6e-4671-94b5-7506a56dfb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:52.429772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:52.497884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:52.537386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1fb9c0-ba9d-45a7-b6a7-49d7990d6029 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:52.538925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8a1fb9c0-ba9d-45a7-b6a7-49d7990d6029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:52.641381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1fb9c0-ba9d-45a7-b6a7-49d7990d6029 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:52.669092 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:52.714830 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572232c-017e-4ef6-81aa-cba05c55b1dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:52.715457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572232c-017e-4ef6-81aa-cba05c55b1dd service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:52.715739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572232c-017e-4ef6-81aa-cba05c55b1dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:52.716051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572232c-017e-4ef6-81aa-cba05c55b1dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:52.717919 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2572232c-017e-4ef6-81aa-cba05c55b1dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:52.822026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572232c-017e-4ef6-81aa-cba05c55b1dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:52.823397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572232c-017e-4ef6-81aa-cba05c55b1dd service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:52.841352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 730/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:54.886184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4161eb-6537-481e-bf63-3d48f993091c None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:54.886689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4161eb-6537-481e-bf63-3d48f993091c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:54.886689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4161eb-6537-481e-bf63-3d48f993091c None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:54.888042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:54.893327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843f3d8-f2be-48e3-9c86-e876d162635d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:54.894294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843f3d8-f2be-48e3-9c86-e876d162635d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:54.894469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843f3d8-f2be-48e3-9c86-e876d162635d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:54.958508 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7843f3d8-f2be-48e3-9c86-e876d162635d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:54.959729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7843f3d8-f2be-48e3-9c86-e876d162635d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:54.968011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:55.053368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:55.091579 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4161eb-6537-481e-bf63-3d48f993091c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:55.093501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2f4161eb-6537-481e-bf63-3d48f993091c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:55.195351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4161eb-6537-481e-bf63-3d48f993091c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:55.249693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:55.268109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c4b39-f8f6-41ef-ab4e-52efc37b3b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:55.268790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c4b39-f8f6-41ef-ab4e-52efc37b3b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:55.269068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c4b39-f8f6-41ef-ab4e-52efc37b3b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:55.269335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c4b39-f8f6-41ef-ab4e-52efc37b3b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:55.270931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ff1c4b39-f8f6-41ef-ab4e-52efc37b3b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:55.376687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c4b39-f8f6-41ef-ab4e-52efc37b3b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:55.378691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c4b39-f8f6-41ef-ab4e-52efc37b3b4a service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:55.398007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 730/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:16:57.439354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c342d49-7e85-41df-9c70-e9c405155fc3 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:57.439961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c342d49-7e85-41df-9c70-e9c405155fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:57.439961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c342d49-7e85-41df-9c70-e9c405155fc3 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:57.441997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:16:57.447325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db6b9-b3bc-4436-909c-456b970d74f5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:57.447725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db6b9-b3bc-4436-909c-456b970d74f5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:57.447897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837db6b9-b3bc-4436-909c-456b970d74f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:57.512085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-837db6b9-b3bc-4436-909c-456b970d74f5 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:16:57.513278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-837db6b9-b3bc-4436-909c-456b970d74f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:57.550213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:57.609254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:16:57.648139 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c342d49-7e85-41df-9c70-e9c405155fc3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:16:57.649690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6c342d49-7e85-41df-9c70-e9c405155fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:57.750826 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c342d49-7e85-41df-9c70-e9c405155fc3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:16:57.788552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:16:57.822987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17844689-9774-4688-a211-08792524db9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:16:57.823628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17844689-9774-4688-a211-08792524db9d service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:16:57.823935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17844689-9774-4688-a211-08792524db9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:16:57.824451 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17844689-9774-4688-a211-08792524db9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:16:57.826856 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-17844689-9774-4688-a211-08792524db9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:16:57.931916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17844689-9774-4688-a211-08792524db9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:16:57.933367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17844689-9774-4688-a211-08792524db9d service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:16:57.951616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 733/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:17:00.006724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d2955-f417-4215-b2f7-526a3f5ab00a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:00.007338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d2955-f417-4215-b2f7-526a3f5ab00a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:00.007338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d2955-f417-4215-b2f7-526a3f5ab00a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:00.008644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:17:00.013869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db76667-9958-41d1-af97-0d134dfd3c0c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:00.014320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db76667-9958-41d1-af97-0d134dfd3c0c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:00.014466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db76667-9958-41d1-af97-0d134dfd3c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:00.084976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7db76667-9958-41d1-af97-0d134dfd3c0c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:17:00.086203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7db76667-9958-41d1-af97-0d134dfd3c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:00.104763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:00.182847 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:17:00.219236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d2955-f417-4215-b2f7-526a3f5ab00a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:17:00.220800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6f9d2955-f417-4215-b2f7-526a3f5ab00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:00.323547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d2955-f417-4215-b2f7-526a3f5ab00a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:17:00.403659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:00.412438 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5eb71-86f5-45df-9e78-de0b6e7aad89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:17:00.415090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5eb71-86f5-45df-9e78-de0b6e7aad89 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:00.415090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5eb71-86f5-45df-9e78-de0b6e7aad89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:00.415090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5eb71-86f5-45df-9e78-de0b6e7aad89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:00.415377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2ba5eb71-86f5-45df-9e78-de0b6e7aad89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:00.523692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5eb71-86f5-45df-9e78-de0b6e7aad89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:17:00.525146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5eb71-86f5-45df-9e78-de0b6e7aad89 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:17:00.542564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 733/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:17:02.555935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af807bf2-2c54-4452-bf9d-fed3650d0cb5 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:02.556470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af807bf2-2c54-4452-bf9d-fed3650d0cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:02.556470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af807bf2-2c54-4452-bf9d-fed3650d0cb5 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:02.557777 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:17:02.563005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140704-1ccc-40cc-a90b-5fd97ac23aae None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:02.563426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140704-1ccc-40cc-a90b-5fd97ac23aae None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:02.563571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140704-1ccc-40cc-a90b-5fd97ac23aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:02.631016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ab140704-1ccc-40cc-a90b-5fd97ac23aae None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:17:02.632247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ab140704-1ccc-40cc-a90b-5fd97ac23aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:02.678872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:02.726234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:17:02.762780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af807bf2-2c54-4452-bf9d-fed3650d0cb5 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:17:02.764324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-af807bf2-2c54-4452-bf9d-fed3650d0cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:02.866100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af807bf2-2c54-4452-bf9d-fed3650d0cb5 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:17:02.913030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:02.937875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94692ce8-fdf7-4978-bd01-58517fb402f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:17:02.938495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94692ce8-fdf7-4978-bd01-58517fb402f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:02.938774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94692ce8-fdf7-4978-bd01-58517fb402f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:02.939047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94692ce8-fdf7-4978-bd01-58517fb402f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:02.940778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-94692ce8-fdf7-4978-bd01-58517fb402f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:03.047324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94692ce8-fdf7-4978-bd01-58517fb402f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:17:03.048791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94692ce8-fdf7-4978-bd01-58517fb402f8 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:17:03.067200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 736/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:17:12.724403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=22 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:13.064407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=66 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:22.590062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d480564-c652-4bc0-8cc0-e5a8ea22aa39 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:22.590543 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d480564-c652-4bc0-8cc0-e5a8ea22aa39 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:22.590639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d480564-c652-4bc0-8cc0-e5a8ea22aa39 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:22.591999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 735/1471] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:17:22.598001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c97d07-813f-42f0-b5db-70b1975d9a77 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:22.598438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c97d07-813f-42f0-b5db-70b1975d9a77 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:22.598577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c97d07-813f-42f0-b5db-70b1975d9a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:22.664282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e7c97d07-813f-42f0-b5db-70b1975d9a77 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:17:22.665504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e7c97d07-813f-42f0-b5db-70b1975d9a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:22.760779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 737/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:17:22.843346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318595a9-38bf-4cf3-aa74-5af84e83fee2 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:22.843640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318595a9-38bf-4cf3-aa74-5af84e83fee2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:22.843963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318595a9-38bf-4cf3-aa74-5af84e83fee2 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:22.845305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 736/1473] 10.222.0.25 () {54 vars in 966 bytes} [Sat Aug 6 01:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:17:22.850170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427c8e-5d90-46f2-bb0a-b0fa7103c58b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:22.850465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427c8e-5d90-46f2-bb0a-b0fa7103c58b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:22.850747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427c8e-5d90-46f2-bb0a-b0fa7103c58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:22.925049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e9427c8e-5d90-46f2-bb0a-b0fa7103c58b None None] MFA Rules not processed for user `9a234a96eccd4f38b0213d7d0461ad24`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:17:22.926258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e9427c8e-5d90-46f2-bb0a-b0fa7103c58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:22.947305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:23.001937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 738/1474] 10.222.0.25 () {58 vars in 1063 bytes} [Sat Aug 6 01:17:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:17:23.009997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318595a9-38bf-4cf3-aa74-5af84e83fee2 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:17:23.011502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-318595a9-38bf-4cf3-aa74-5af84e83fee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:23.112983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318595a9-38bf-4cf3-aa74-5af84e83fee2 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:17:23.185109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c9877-a593-401b-b519-d2904f7c9627 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a234a96eccd4f38b0213d7d0461ad24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:17:23.185726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c9877-a593-401b-b519-d2904f7c9627 service glance] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:23.186010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c9877-a593-401b-b519-d2904f7c9627 service glance] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:23.186288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c9877-a593-401b-b519-d2904f7c9627 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:23.187954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-522c9877-a593-401b-b519-d2904f7c9627 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:23.242656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:23.297137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c9877-a593-401b-b519-d2904f7c9627 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:17:23.298602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c9877-a593-401b-b519-d2904f7c9627 service glance] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:17:23.317306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 737/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:17:25.406673 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e4685-ac8b-4d53-aa9b-a72a53e195e8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:25.407266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e4685-ac8b-4d53-aa9b-a72a53e195e8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:25.407266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e4685-ac8b-4d53-aa9b-a72a53e195e8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:25.408613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 739/1476] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:17:25.414559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1648d3c-3ee5-4235-be65-683c32d6e440 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:25.414966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1648d3c-3ee5-4235-be65-683c32d6e440 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:25.415111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1648d3c-3ee5-4235-be65-683c32d6e440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:25.479906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d1648d3c-3ee5-4235-be65-683c32d6e440 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:17:25.481114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d1648d3c-3ee5-4235-be65-683c32d6e440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:25.582698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 738/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:17:25.606391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e4685-ac8b-4d53-aa9b-a72a53e195e8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:17:25.607938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2c0e4685-ac8b-4d53-aa9b-a72a53e195e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:25.731434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e4685-ac8b-4d53-aa9b-a72a53e195e8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:17:25.778558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:25.822358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0568c-9820-4b43-8982-f0ee5107799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:17:25.822972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0568c-9820-4b43-8982-f0ee5107799b admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:25.823248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0568c-9820-4b43-8982-f0ee5107799b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:25.823517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0568c-9820-4b43-8982-f0ee5107799b admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:25.824253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0568c-9820-4b43-8982-f0ee5107799b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:17:25.828193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-3ba0568c-9820-4b43-8982-f0ee5107799b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:17:25.828193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 06 01:17:25.828193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: Aug 06 01:17:25.828193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba0568c-9820-4b43-8982-f0ee5107799b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:17:25.846981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 740/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Sat Aug 6 01:17:25 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:17:27.906930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f7392-b2c8-493a-aa49-590a8321bbf4 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:27.907423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f7392-b2c8-493a-aa49-590a8321bbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:27.907530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f7392-b2c8-493a-aa49-590a8321bbf4 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:27.908962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 739/1479] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:17:27.914027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0588a73-5625-4df6-8780-305c6782f0cb None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:27.914461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0588a73-5625-4df6-8780-305c6782f0cb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:27.914612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0588a73-5625-4df6-8780-305c6782f0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:27.978674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b0588a73-5625-4df6-8780-305c6782f0cb None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:17:27.979911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b0588a73-5625-4df6-8780-305c6782f0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:28.072681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 741/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:17:28.115538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f7392-b2c8-493a-aa49-590a8321bbf4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:17:28.117235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2f9f7392-b2c8-493a-aa49-590a8321bbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:28.126984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:28.260339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f7392-b2c8-493a-aa49-590a8321bbf4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:17:28.337283 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:28.370019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85479517-5485-4e81-9320-b9c3fcbf3717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:17:28.370631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85479517-5485-4e81-9320-b9c3fcbf3717 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:28.370912 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85479517-5485-4e81-9320-b9c3fcbf3717 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:28.371185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85479517-5485-4e81-9320-b9c3fcbf3717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:28.372816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-85479517-5485-4e81-9320-b9c3fcbf3717 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:28.460105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85479517-5485-4e81-9320-b9c3fcbf3717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:17:28.461755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85479517-5485-4e81-9320-b9c3fcbf3717 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:17:28.480243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 740/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:17:30.703369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68369696-0b4a-4017-b209-f57789132289 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:30.703981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68369696-0b4a-4017-b209-f57789132289 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:30.703981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68369696-0b4a-4017-b209-f57789132289 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:30.705376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 742/1482] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:17:30.711123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89161212-fd12-4939-ac67-f5bc02272a74 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:30.711569 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89161212-fd12-4939-ac67-f5bc02272a74 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:30.711673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89161212-fd12-4939-ac67-f5bc02272a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:30.777151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-89161212-fd12-4939-ac67-f5bc02272a74 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:17:30.778360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-89161212-fd12-4939-ac67-f5bc02272a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:30.825687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:30.876438 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 741/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:17:30.913512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68369696-0b4a-4017-b209-f57789132289 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:17:30.915035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-68369696-0b4a-4017-b209-f57789132289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:30.938550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:31.019729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68369696-0b4a-4017-b209-f57789132289 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:17:31.096468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7824e-3fa4-47b1-bcb9-6ef8b2952ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:17:31.097095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7824e-3fa4-47b1-bcb9-6ef8b2952ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:31.097376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7824e-3fa4-47b1-bcb9-6ef8b2952ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:31.097645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7824e-3fa4-47b1-bcb9-6ef8b2952ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:31.099281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-56c7824e-3fa4-47b1-bcb9-6ef8b2952ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:31.181085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:31.214852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7824e-3fa4-47b1-bcb9-6ef8b2952ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:17:31.216340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7824e-3fa4-47b1-bcb9-6ef8b2952ab2 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:17:31.233902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 743/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:17:33.324702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dae31-beea-4127-9082-7dc3cd379f25 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:33.325294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dae31-beea-4127-9082-7dc3cd379f25 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:33.325294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dae31-beea-4127-9082-7dc3cd379f25 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:33.326577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 742/1485] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:17:33.332243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15408680-747c-4db9-998f-435ee0f425fe None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:33.332640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15408680-747c-4db9-998f-435ee0f425fe None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:33.332800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15408680-747c-4db9-998f-435ee0f425fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:33.400785 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-15408680-747c-4db9-998f-435ee0f425fe None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:17:33.402029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-15408680-747c-4db9-998f-435ee0f425fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:33.497273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 744/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:17:33.535579 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dae31-beea-4127-9082-7dc3cd379f25 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:17:33.537204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7e5dae31-beea-4127-9082-7dc3cd379f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:33.644639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dae31-beea-4127-9082-7dc3cd379f25 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:17:33.713305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:33.715008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c5536-0050-4dc2-b3fa-d892dca06b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:17:33.715614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c5536-0050-4dc2-b3fa-d892dca06b0a service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:33.715926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c5536-0050-4dc2-b3fa-d892dca06b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:33.716199 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c5536-0050-4dc2-b3fa-d892dca06b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:33.717801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-110c5536-0050-4dc2-b3fa-d892dca06b0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:33.823770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c5536-0050-4dc2-b3fa-d892dca06b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:17:33.825209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c5536-0050-4dc2-b3fa-d892dca06b0a service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:17:33.842492 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 743/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:17:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:17:35.941770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7044a31-2420-4f04-8d83-3c9be478be5b None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:35.942257 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7044a31-2420-4f04-8d83-3c9be478be5b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:35.942372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7044a31-2420-4f04-8d83-3c9be478be5b None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:35.943731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 745/1488] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:17:35.949110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a6cde-506f-4f34-8909-8a65e40c3360 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:35.949504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a6cde-506f-4f34-8909-8a65e40c3360 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:35.949677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a6cde-506f-4f34-8909-8a65e40c3360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:36.015239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2a8a6cde-506f-4f34-8909-8a65e40c3360 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:17:36.016481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2a8a6cde-506f-4f34-8909-8a65e40c3360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:36.069386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:36.113661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 744/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:17:36.154853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7044a31-2420-4f04-8d83-3c9be478be5b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:17:36.156449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f7044a31-2420-4f04-8d83-3c9be478be5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:36.192807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:36.260888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7044a31-2420-4f04-8d83-3c9be478be5b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:17:36.334303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23138913-acda-436a-9b8d-4249de0a720b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:17:36.334924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138913-acda-436a-9b8d-4249de0a720b service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:36.335204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138913-acda-436a-9b8d-4249de0a720b service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:36.335481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138913-acda-436a-9b8d-4249de0a720b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:36.337138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-23138913-acda-436a-9b8d-4249de0a720b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:36.415355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:36.445905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23138913-acda-436a-9b8d-4249de0a720b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:17:36.447487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23138913-acda-436a-9b8d-4249de0a720b service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:17:36.465853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 746/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:17:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:17:38.885735 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c2151-a2ec-4d16-83fa-747c7bf68959 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:38.886256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c2151-a2ec-4d16-83fa-747c7bf68959 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:38.886256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c2151-a2ec-4d16-83fa-747c7bf68959 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:38.887541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 745/1491] 10.222.0.25 () {54 vars in 927 bytes} [Sat Aug 6 01:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:17:38.892710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5aeb9-253d-409d-9b3a-ce5be0f5b8e9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:38.893130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5aeb9-253d-409d-9b3a-ce5be0f5b8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:38.893917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5aeb9-253d-409d-9b3a-ce5be0f5b8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:38.958509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-39b5aeb9-253d-409d-9b3a-ce5be0f5b8e9 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:17:38.959703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-39b5aeb9-253d-409d-9b3a-ce5be0f5b8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:39.052862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 747/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Sat Aug 6 01:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:17:39.091206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c2151-a2ec-4d16-83fa-747c7bf68959 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:17:39.092744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-924c2151-a2ec-4d16-83fa-747c7bf68959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:39.195175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c2151-a2ec-4d16-83fa-747c7bf68959 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:17:39.265859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa80f8d-340d-442e-94bf-1176e88ac3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:17:39.266475 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa80f8d-340d-442e-94bf-1176e88ac3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:17:39.266753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa80f8d-340d-442e-94bf-1176e88ac3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:17:39.267027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa80f8d-340d-442e-94bf-1176e88ac3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:17:39.269305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2aa80f8d-340d-442e-94bf-1176e88ac3e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:17:39.278137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:39.376036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa80f8d-340d-442e-94bf-1176e88ac3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:17:39.377477 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa80f8d-340d-442e-94bf-1176e88ac3e0 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:17:39.396257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 746/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:17:49.051054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=82 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:17:49.394160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=50 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:18:56.942317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf2fbe-685c-4de5-8ddc-e5cb43087df0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:56.942897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf2fbe-685c-4de5-8ddc-e5cb43087df0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:56.942897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf2fbe-685c-4de5-8ddc-e5cb43087df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:57.016008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1caf2fbe-685c-4de5-8ddc-e5cb43087df0 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:18:57.017216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1caf2fbe-685c-4de5-8ddc-e5cb43087df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:57.111113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 748/1494] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:18:57.119840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cbb40-8c43-4707-9735-a88bc39a4cf1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:57.120332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cbb40-8c43-4707-9735-a88bc39a4cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:57.120439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cbb40-8c43-4707-9735-a88bc39a4cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:57.194620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-163cbb40-8c43-4707-9735-a88bc39a4cf1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:18:57.195853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-163cbb40-8c43-4707-9735-a88bc39a4cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:57.291851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 747/1495] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:18:57.296431 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf2fbe-685c-4de5-8ddc-e5cb43087df0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:18:57.297622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1caf2fbe-685c-4de5-8ddc-e5cb43087df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:57.309402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf2fbe-685c-4de5-8ddc-e5cb43087df0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:18:57.369407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:18:57.402257 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:18:57.402873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:57.403154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:57.403424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:57.404137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:18:57.404436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:18:57.405450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:18:57.405450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 06 01:18:57.405450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: Aug 06 01:18:57.406382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:18:57.418300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 749/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:18:57.426493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cbb40-8c43-4707-9735-a88bc39a4cf1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:18:57.435535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cbb40-8c43-4707-9735-a88bc39a4cf1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:18:57.495011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:18:57.527252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd85239-58eb-4b97-8858-321d3a884fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:18:57.527896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd85239-58eb-4b97-8858-321d3a884fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:57.528176 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd85239-58eb-4b97-8858-321d3a884fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:57.528449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd85239-58eb-4b97-8858-321d3a884fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:57.529108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd85239-58eb-4b97-8858-321d3a884fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:18:57.530799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-2cd85239-58eb-4b97-8858-321d3a884fcf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:18:57.530799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 06 01:18:57.530799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: Aug 06 01:18:57.532056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd85239-58eb-4b97-8858-321d3a884fcf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:18:57.672762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 748/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:18:57 2022] POST /identity/v3/projects => generated 379 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:18:57.677353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:18:57.678911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:57.714018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fbdba-63eb-4ee3-9ddb-8bf5124857a9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:18:57.806013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a175d5d-3cdc-4c64-ae17-dc30af257f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:18:57.806562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a175d5d-3cdc-4c64-ae17-dc30af257f36 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:57.806841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a175d5d-3cdc-4c64-ae17-dc30af257f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:57.807113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a175d5d-3cdc-4c64-ae17-dc30af257f36 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:57.807768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a175d5d-3cdc-4c64-ae17-dc30af257f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:18:57.808816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-7a175d5d-3cdc-4c64-ae17-dc30af257f36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:18:57.808816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 06 01:18:57.808816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: Aug 06 01:18:57.810032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a175d5d-3cdc-4c64-ae17-dc30af257f36 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:18:57.900138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 750/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:18:57 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:18:57.904289 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd85239-58eb-4b97-8858-321d3a884fcf admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:18:57.905785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2cd85239-58eb-4b97-8858-321d3a884fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:57.965793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd85239-58eb-4b97-8858-321d3a884fcf admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:18:58.045383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:18:58.057856 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4286c5d-3d52-4472-a94c-fd343814c209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:18:58.058401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4286c5d-3d52-4472-a94c-fd343814c209 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:58.058678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4286c5d-3d52-4472-a94c-fd343814c209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:58.058950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4286c5d-3d52-4472-a94c-fd343814c209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:58.059643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4286c5d-3d52-4472-a94c-fd343814c209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:18:58.060669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-c4286c5d-3d52-4472-a94c-fd343814c209 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:18:58.060669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 06 01:18:58.060669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: Aug 06 01:18:58.061548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4286c5d-3d52-4472-a94c-fd343814c209 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:18:58.074894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 749/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:18:58.079077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a175d5d-3cdc-4c64-ae17-dc30af257f36 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:18:58.080620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7a175d5d-3cdc-4c64-ae17-dc30af257f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:58.092573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a175d5d-3cdc-4c64-ae17-dc30af257f36 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:18:58.117496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:18:58.186075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b1bb3-279a-4546-b53e-81eccb0e58b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:18:58.186706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b1bb3-279a-4546-b53e-81eccb0e58b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:58.186998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b1bb3-279a-4546-b53e-81eccb0e58b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:58.187276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b1bb3-279a-4546-b53e-81eccb0e58b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:58.187995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b1bb3-279a-4546-b53e-81eccb0e58b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:18:58.189025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-cb9b1bb3-279a-4546-b53e-81eccb0e58b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:18:58.189025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 06 01:18:58.189025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: Aug 06 01:18:58.189919 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b1bb3-279a-4546-b53e-81eccb0e58b2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:18:58.201992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 751/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:18:58.206910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4286c5d-3d52-4472-a94c-fd343814c209 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:18:58.208453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c4286c5d-3d52-4472-a94c-fd343814c209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:58.241965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4286c5d-3d52-4472-a94c-fd343814c209 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:18:58.334114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca01ec6-02a0-4631-8841-e39e67e44534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:18:58.334702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca01ec6-02a0-4631-8841-e39e67e44534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:58.334985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca01ec6-02a0-4631-8841-e39e67e44534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:58.335262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca01ec6-02a0-4631-8841-e39e67e44534 admin admin] PATH_INFO: `/v3/projects/4691c91026444186aebefd6bf1178bb8/users/98aacbec406548f29344c6d8f448992f/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:58.357264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca01ec6-02a0-4631-8841-e39e67e44534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4691c91026444186aebefd6bf1178bb8, user_id=98aacbec406548f29344c6d8f448992f, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:18:58.358281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-3ca01ec6-02a0-4631-8841-e39e67e44534 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:18:58.358281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 06 01:18:58.358281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: Aug 06 01:18:58.360247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca01ec6-02a0-4631-8841-e39e67e44534 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:18:58.460317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 750/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:18:58 2022] PUT /identity/v3/projects/4691c91026444186aebefd6bf1178bb8/users/98aacbec406548f29344c6d8f448992f/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:18:58.472655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf846647-44de-4c13-89cb-223b81ff2ae3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:58.472942 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf846647-44de-4c13-89cb-223b81ff2ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:58.473223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf846647-44de-4c13-89cb-223b81ff2ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:58.528386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bf846647-44de-4c13-89cb-223b81ff2ae3 None None] MFA Rules not processed for user `98aacbec406548f29344c6d8f448992f`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:18:58.529583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bf846647-44de-4c13-89cb-223b81ff2ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:58.606351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 752/1502] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:18:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:18:58.619368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4038-e30b-48bc-a036-2e3522be6d42 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:58.619658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4038-e30b-48bc-a036-2e3522be6d42 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:58.619965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4038-e30b-48bc-a036-2e3522be6d42 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:58.621316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 751/1503] 10.222.0.25 () {54 vars in 964 bytes} [Sat Aug 6 01:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:18:58.625954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe624d-8955-473b-a562-62b0f0437dad None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:58.626245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe624d-8955-473b-a562-62b0f0437dad None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:58.626522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe624d-8955-473b-a562-62b0f0437dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:58.657605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:18:58.706123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b3fe624d-8955-473b-a562-62b0f0437dad None None] MFA Rules not processed for user `ddd0e588d52e40f08a2dc1062ecd4232`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:18:58.707333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b3fe624d-8955-473b-a562-62b0f0437dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:58.780938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 753/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Sat Aug 6 01:18:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:18:58.789328 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4038-e30b-48bc-a036-2e3522be6d42 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:18:58.790833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-86ea4038-e30b-48bc-a036-2e3522be6d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:58.839422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:18:58.891879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4038-e30b-48bc-a036-2e3522be6d42 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:18:58.962582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1d197-d7b0-48a3-8ea6-f36644c03975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:18:58.963194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d197-d7b0-48a3-8ea6-f36644c03975 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:58.963475 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d197-d7b0-48a3-8ea6-f36644c03975 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:58.963750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d197-d7b0-48a3-8ea6-f36644c03975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:58.965377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-09f1d197-d7b0-48a3-8ea6-f36644c03975 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:59.063224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1d197-d7b0-48a3-8ea6-f36644c03975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:18:59.064683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1d197-d7b0-48a3-8ea6-f36644c03975 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:18:59.068781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:18:59.082877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 752/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:18:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:18:59.127110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029f07e-5519-4622-b8e1-465b71a36109 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:59.127545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029f07e-5519-4622-b8e1-465b71a36109 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:59.127664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029f07e-5519-4622-b8e1-465b71a36109 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:59.129566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 754/1506] 10.222.0.25 () {52 vars in 830 bytes} [Sat Aug 6 01:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:18:59.133103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1d197-d7b0-48a3-8ea6-f36644c03975 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:18:59.134298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-09f1d197-d7b0-48a3-8ea6-f36644c03975 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:59.230597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1d197-d7b0-48a3-8ea6-f36644c03975 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:18:59.322029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:18:59.322716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:59.322996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:59.323271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] PATH_INFO: `/v3/users/98aacbec406548f29344c6d8f448992f` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:59.335139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98aacbec406548f29344c6d8f448992f)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:18:59.336183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:18:59.336183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 06 01:18:59.336183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: Aug 06 01:18:59.337395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:18:59.347508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:18:59.554163 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98aacbec406548f29344c6d8f448992f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:18:59.568244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98aacbec406548f29344c6d8f448992f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:18:59.641970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98aacbec406548f29344c6d8f448992f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:18:59.642509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98aacbec406548f29344c6d8f448992f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:18:59.667662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98aacbec406548f29344c6d8f448992f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:18:59.673158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 753/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:18:59 2022] DELETE /identity/v3/users/98aacbec406548f29344c6d8f448992f => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:18:59.677643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544201f-ec39-4f26-9339-6bfc79274377 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:18:59.677931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544201f-ec39-4f26-9339-6bfc79274377 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:18:59.678238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544201f-ec39-4f26-9339-6bfc79274377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:18:59.741119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8544201f-ec39-4f26-9339-6bfc79274377 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:18:59.742336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8544201f-ec39-4f26-9339-6bfc79274377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:59.813987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:18:59.826351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 755/1508] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:18:59.836615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:18:59.837838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:18:59.938274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901c203-bc57-4e3c-b579-b3c543321d04 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:18:59.993235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:00.011936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f28f1-e1fb-45d6-aa18-cc74ff6e95d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:00.012582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f28f1-e1fb-45d6-aa18-cc74ff6e95d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:00.012860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f28f1-e1fb-45d6-aa18-cc74ff6e95d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:00.013134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f28f1-e1fb-45d6-aa18-cc74ff6e95d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:00.014744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ef8f28f1-e1fb-45d6-aa18-cc74ff6e95d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:00.131425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f28f1-e1fb-45d6-aa18-cc74ff6e95d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:00.132814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f28f1-e1fb-45d6-aa18-cc74ff6e95d5 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:00.150926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 754/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:19:00.599490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544201f-ec39-4f26-9339-6bfc79274377 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:19:00.600817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8544201f-ec39-4f26-9339-6bfc79274377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:19:00.612603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544201f-ec39-4f26-9339-6bfc79274377 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:19:00.706360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:19:00.707622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:19:00.707943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:19:00.708218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] PATH_INFO: `/v3/projects/4691c91026444186aebefd6bf1178bb8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:19:00.717907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4691c91026444186aebefd6bf1178bb8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:19:00.718940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:19:00.718940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 06 01:19:00.718940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: Aug 06 01:19:00.720184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:19:00.822874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4691c91026444186aebefd6bf1178bb8', 'request_id': 'req-6049055e-087d-48d5-b62f-302f21b4b578'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:19:00.828975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4691c91026444186aebefd6bf1178bb8', 'request_id': 'req-6049055e-087d-48d5-b62f-302f21b4b578'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:19:00.898749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4691c91026444186aebefd6bf1178bb8', 'request_id': 'req-6049055e-087d-48d5-b62f-302f21b4b578'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:19:00.929592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] The token cache is being invalidate because project 4691c91026444186aebefd6bf1178bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:19:00.929882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:19:00.930407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6049055e-087d-48d5-b62f-302f21b4b578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:19:00.931812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 756/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:19:00 2022] DELETE /identity/v3/projects/4691c91026444186aebefd6bf1178bb8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:19:10.148662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=62 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:19:10.933618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:28.235164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04faec-64c3-4a31-bcb7-7e2465fe21cb None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.235748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04faec-64c3-4a31-bcb7-7e2465fe21cb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.235748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04faec-64c3-4a31-bcb7-7e2465fe21cb None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.237037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:28.242314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049cf7b-3362-4477-806b-f174ea33c61a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.242733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049cf7b-3362-4477-806b-f174ea33c61a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.242858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049cf7b-3362-4477-806b-f174ea33c61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.322673 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8049cf7b-3362-4477-806b-f174ea33c61a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:28.323986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8049cf7b-3362-4477-806b-f174ea33c61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:28.429529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:20:28.582870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04faec-64c3-4a31-bcb7-7e2465fe21cb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:28.585540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3c04faec-64c3-4a31-bcb7-7e2465fe21cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:28.701335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04faec-64c3-4a31-bcb7-7e2465fe21cb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:28.706639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f7c8bd-5390-4e50-84f5-6ea2b9b40d70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:28.707248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7c8bd-5390-4e50-84f5-6ea2b9b40d70 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:28.707530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7c8bd-5390-4e50-84f5-6ea2b9b40d70 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:28.707833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7c8bd-5390-4e50-84f5-6ea2b9b40d70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:28.716751 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7c8bd-5390-4e50-84f5-6ea2b9b40d70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:28.718379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f7c8bd-5390-4e50-84f5-6ea2b9b40d70 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:28.728614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 756/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:20:29.542797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f0d3c-bfc6-40e6-9fe6-3a3dbb065378 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.543336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f0d3c-bfc6-40e6-9fe6-3a3dbb065378 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.543336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f0d3c-bfc6-40e6-9fe6-3a3dbb065378 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.544728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:29.549385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1c4bc-970b-4de4-b1c0-a6c789d304c3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.549809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1c4bc-970b-4de4-b1c0-a6c789d304c3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.549944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1c4bc-970b-4de4-b1c0-a6c789d304c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.569848 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8fb1c4bc-970b-4de4-b1c0-a6c789d304c3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:29.571042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8fb1c4bc-970b-4de4-b1c0-a6c789d304c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:29.578966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:20:29.730515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f0d3c-bfc6-40e6-9fe6-3a3dbb065378 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:29.745326 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f0d3c-bfc6-40e6-9fe6-3a3dbb065378 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:29.750596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945b2d4-938e-493b-8c77-0213fe424ce0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:29.751198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945b2d4-938e-493b-8c77-0213fe424ce0 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:29.751477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945b2d4-938e-493b-8c77-0213fe424ce0 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:29.751741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945b2d4-938e-493b-8c77-0213fe424ce0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:29.759604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945b2d4-938e-493b-8c77-0213fe424ce0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:29.761219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945b2d4-938e-493b-8c77-0213fe424ce0 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:29.770372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 759/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:20:30.146397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906ccf-ef49-4f9e-82d7-aa33618214ed None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:30.146397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906ccf-ef49-4f9e-82d7-aa33618214ed None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:30.146397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906ccf-ef49-4f9e-82d7-aa33618214ed None None] PATH_INFO: `/v3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:30.147896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 758/1517] 10.222.0.36 () {54 vars in 925 bytes} [Sat Aug 6 01:20:30 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:20:30.153960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefca28-9097-4a21-bbd6-e5e8ad296626 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:30.154252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefca28-9097-4a21-bbd6-e5e8ad296626 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:30.154576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefca28-9097-4a21-bbd6-e5e8ad296626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:30.212421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7fefca28-9097-4a21-bbd6-e5e8ad296626 None None] MFA Rules not processed for user `8361f2f9c70b4a3288a187cb2d97d415`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:30.213629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7fefca28-9097-4a21-bbd6-e5e8ad296626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:30.230547 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:30.288442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 760/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Sat Aug 6 01:20:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:20:39.364195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd340d8d-f4b5-4735-9266-3f429e056f1a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.364746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd340d8d-f4b5-4735-9266-3f429e056f1a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.364746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd340d8d-f4b5-4735-9266-3f429e056f1a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.366041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 759/1519] 10.222.0.25 () {54 vars in 961 bytes} [Sat Aug 6 01:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:39.371232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b9e72-2286-4e82-872d-3eacda433a61 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.371630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b9e72-2286-4e82-872d-3eacda433a61 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.371822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b9e72-2286-4e82-872d-3eacda433a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.391328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f24b9e72-2286-4e82-872d-3eacda433a61 None None] MFA Rules not processed for user `8361f2f9c70b4a3288a187cb2d97d415`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:39.392557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f24b9e72-2286-4e82-872d-3eacda433a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:39.401651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 761/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Sat Aug 6 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:20:39.408792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd340d8d-f4b5-4735-9266-3f429e056f1a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:39.421919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd340d8d-f4b5-4735-9266-3f429e056f1a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:39.426843 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f0979-def5-47f2-929b-4037eb37bc59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:39.427442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0979-def5-47f2-929b-4037eb37bc59 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:39.427728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0979-def5-47f2-929b-4037eb37bc59 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:39.428043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0979-def5-47f2-929b-4037eb37bc59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:39.436169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f0979-def5-47f2-929b-4037eb37bc59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:39.437742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f0979-def5-47f2-929b-4037eb37bc59 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:39.447697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 760/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:20:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:20:40.998162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ec7c6-dcdb-47fa-9f2e-c6ce6af2582d None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:40.998730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ec7c6-dcdb-47fa-9f2e-c6ce6af2582d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:40.998730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ec7c6-dcdb-47fa-9f2e-c6ce6af2582d None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.000112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:41.004968 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28bbf9-a6a3-479a-ba9b-fade594e2477 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.005397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28bbf9-a6a3-479a-ba9b-fade594e2477 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.005528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28bbf9-a6a3-479a-ba9b-fade594e2477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.025539 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7d28bbf9-a6a3-479a-ba9b-fade594e2477 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:41.026726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7d28bbf9-a6a3-479a-ba9b-fade594e2477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:41.034485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:20:41.187200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ec7c6-dcdb-47fa-9f2e-c6ce6af2582d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:41.202086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ec7c6-dcdb-47fa-9f2e-c6ce6af2582d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:41.207539 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae247ab-6d03-448c-a14c-0874934a9a8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:41.208184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae247ab-6d03-448c-a14c-0874934a9a8d service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:41.208469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae247ab-6d03-448c-a14c-0874934a9a8d service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:41.208744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae247ab-6d03-448c-a14c-0874934a9a8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:41.216853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae247ab-6d03-448c-a14c-0874934a9a8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:41.220336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae247ab-6d03-448c-a14c-0874934a9a8d service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:41.228046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 763/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:20:42.050473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b08412-995b-4f8e-8739-45cd28ac2cde None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.051010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b08412-995b-4f8e-8739-45cd28ac2cde None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.051010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b08412-995b-4f8e-8739-45cd28ac2cde None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.052394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 01:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:42.057266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2458c3-61bf-42b0-a66e-b780cc2fbbb0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.057668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2458c3-61bf-42b0-a66e-b780cc2fbbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.057825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2458c3-61bf-42b0-a66e-b780cc2fbbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.076732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8d2458c3-61bf-42b0-a66e-b780cc2fbbb0 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:42.077927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8d2458c3-61bf-42b0-a66e-b780cc2fbbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:42.086182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 01:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:20:42.236264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b08412-995b-4f8e-8739-45cd28ac2cde None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:42.250707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b08412-995b-4f8e-8739-45cd28ac2cde None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:42.256098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c022b-9523-4354-8214-d381b5738cfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:42.256712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c022b-9523-4354-8214-d381b5738cfc service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:42.256966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c022b-9523-4354-8214-d381b5738cfc service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:42.257240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c022b-9523-4354-8214-d381b5738cfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:42.265848 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c022b-9523-4354-8214-d381b5738cfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:42.267425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c022b-9523-4354-8214-d381b5738cfc service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:42.276721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 763/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:20:46.074925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12be97-0e6d-49ae-9bb8-5ecc9a8ab7d8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.075445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12be97-0e6d-49ae-9bb8-5ecc9a8ab7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.075445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12be97-0e6d-49ae-9bb8-5ecc9a8ab7d8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.076904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:46.082303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027d3d-8f20-4e2f-986b-7649ba06f92c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.082724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027d3d-8f20-4e2f-986b-7649ba06f92c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.082857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97027d3d-8f20-4e2f-986b-7649ba06f92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.102587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-97027d3d-8f20-4e2f-986b-7649ba06f92c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:46.103818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-97027d3d-8f20-4e2f-986b-7649ba06f92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:46.111478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:20:46.183440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12be97-0e6d-49ae-9bb8-5ecc9a8ab7d8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:46.185149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bf12be97-0e6d-49ae-9bb8-5ecc9a8ab7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:46.261522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12be97-0e6d-49ae-9bb8-5ecc9a8ab7d8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:46.264938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380df9c6-52c0-4536-8ade-391d653d9542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:46.265556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380df9c6-52c0-4536-8ade-391d653d9542 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:46.265835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380df9c6-52c0-4536-8ade-391d653d9542 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:46.266109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380df9c6-52c0-4536-8ade-391d653d9542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:46.275570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380df9c6-52c0-4536-8ade-391d653d9542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:46.277204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380df9c6-52c0-4536-8ade-391d653d9542 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:46.286575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 766/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:20:48.254051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9634017-4a7a-459f-95f3-75632364d20f None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.254578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9634017-4a7a-459f-95f3-75632364d20f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.254578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9634017-4a7a-459f-95f3-75632364d20f None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.255973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:48.261523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d49300-4034-4fd8-8e18-3c8667c4abc7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.261926 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d49300-4034-4fd8-8e18-3c8667c4abc7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.262076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d49300-4034-4fd8-8e18-3c8667c4abc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.284589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f0d49300-4034-4fd8-8e18-3c8667c4abc7 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:48.285777 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f0d49300-4034-4fd8-8e18-3c8667c4abc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:48.293949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:20:48.331388 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9634017-4a7a-459f-95f3-75632364d20f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:48.344748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9634017-4a7a-459f-95f3-75632364d20f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:48.348299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546f43b-31a7-433a-856f-ed047f3d641b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:48.348907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546f43b-31a7-433a-856f-ed047f3d641b service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:48.349190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546f43b-31a7-433a-856f-ed047f3d641b service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:48.349736 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546f43b-31a7-433a-856f-ed047f3d641b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:48.359312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546f43b-31a7-433a-856f-ed047f3d641b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:48.360760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546f43b-31a7-433a-856f-ed047f3d641b service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:48.370567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 766/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:20:51.796037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbd641-4586-4d2a-94bd-c87ca86d7549 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.796592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbd641-4586-4d2a-94bd-c87ca86d7549 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.796592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbd641-4586-4d2a-94bd-c87ca86d7549 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.797936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:51.803123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d2148-85c8-4ce0-b2a4-e07ece992631 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:51.803549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d2148-85c8-4ce0-b2a4-e07ece992631 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:51.803716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d2148-85c8-4ce0-b2a4-e07ece992631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:51.811750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=81 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:51.824771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a47d2148-85c8-4ce0-b2a4-e07ece992631 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:51.825961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a47d2148-85c8-4ce0-b2a4-e07ece992631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:51.833747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:20:52.137093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbd641-4586-4d2a-94bd-c87ca86d7549 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:52.138763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b5bbd641-4586-4d2a-94bd-c87ca86d7549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:52.208788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:20:52.221978 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbd641-4586-4d2a-94bd-c87ca86d7549 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:52.225895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cc5b5-29eb-4e34-b2cf-e4381b4bcfe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:52.227031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cc5b5-29eb-4e34-b2cf-e4381b4bcfe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:52.227315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cc5b5-29eb-4e34-b2cf-e4381b4bcfe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:52.227594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cc5b5-29eb-4e34-b2cf-e4381b4bcfe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:52.238604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cc5b5-29eb-4e34-b2cf-e4381b4bcfe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:52.240079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cc5b5-29eb-4e34-b2cf-e4381b4bcfe0 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:52.250513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 769/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:20:54.250200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757408-5166-49d3-b8ea-b11cfcfe8c15 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.250708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757408-5166-49d3-b8ea-b11cfcfe8c15 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.250708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757408-5166-49d3-b8ea-b11cfcfe8c15 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.252042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 01:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:20:54.257271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8344115-0ad5-4af1-bee9-bd4d65336973 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.258211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8344115-0ad5-4af1-bee9-bd4d65336973 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.258369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8344115-0ad5-4af1-bee9-bd4d65336973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.281018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e8344115-0ad5-4af1-bee9-bd4d65336973 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:20:54.282215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e8344115-0ad5-4af1-bee9-bd4d65336973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:54.290471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 01:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:20:54.593807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757408-5166-49d3-b8ea-b11cfcfe8c15 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:20:54.595519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b1757408-5166-49d3-b8ea-b11cfcfe8c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:20:54.608931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757408-5166-49d3-b8ea-b11cfcfe8c15 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:20:54.612669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25aeed8-2a88-4c22-a6e1-a498fe99c0ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:20:54.613272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aeed8-2a88-4c22-a6e1-a498fe99c0ee service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:20:54.613550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aeed8-2a88-4c22-a6e1-a498fe99c0ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:20:54.613820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25aeed8-2a88-4c22-a6e1-a498fe99c0ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:20:54.623272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25aeed8-2a88-4c22-a6e1-a498fe99c0ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:20:54.624742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25aeed8-2a88-4c22-a6e1-a498fe99c0ee service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:20:54.634755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 769/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:21:04.275916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=11 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:21:04.629955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=8 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:15.132166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fd95a-cd02-43b2-b2a7-4fae3e6a6b90 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.132880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fd95a-cd02-43b2-b2a7-4fae3e6a6b90 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.132880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fd95a-cd02-43b2-b2a7-4fae3e6a6b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.161325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8e3fd95a-cd02-43b2-b2a7-4fae3e6a6b90 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.163299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8e3fd95a-cd02-43b2-b2a7-4fae3e6a6b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.182127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 771/1540] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:22:15.191456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9899df3-467a-489c-aebb-620baf1cd4c9 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.192545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9899df3-467a-489c-aebb-620baf1cd4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.193054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9899df3-467a-489c-aebb-620baf1cd4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.233018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f9899df3-467a-489c-aebb-620baf1cd4c9 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.234879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f9899df3-467a-489c-aebb-620baf1cd4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.249444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 770/1541] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.254126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fd95a-cd02-43b2-b2a7-4fae3e6a6b90 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:15.265791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fd95a-cd02-43b2-b2a7-4fae3e6a6b90 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.280606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de0bc2-9ca5-432f-84fe-24b9f9f1cf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.281254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0bc2-9ca5-432f-84fe-24b9f9f1cf56 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.281532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0bc2-9ca5-432f-84fe-24b9f9f1cf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.281801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de0bc2-9ca5-432f-84fe-24b9f9f1cf56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.282771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de0bc2-9ca5-432f-84fe-24b9f9f1cf56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:15.283081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de0bc2-9ca5-432f-84fe-24b9f9f1cf56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.284785 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de0bc2-9ca5-432f-84fe-24b9f9f1cf56 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.297731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 772/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:22:15.306139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9899df3-467a-489c-aebb-620baf1cd4c9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:15.323054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9899df3-467a-489c-aebb-620baf1cd4c9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.330514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060d312-e2a4-4740-add2-9fadbe0d062d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.331152 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060d312-e2a4-4740-add2-9fadbe0d062d admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.331427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060d312-e2a4-4740-add2-9fadbe0d062d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.331695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060d312-e2a4-4740-add2-9fadbe0d062d admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.332611 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060d312-e2a4-4740-add2-9fadbe0d062d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.334835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060d312-e2a4-4740-add2-9fadbe0d062d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.465509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559dac3-1739-4b8c-a266-e63852327686 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.465862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559dac3-1739-4b8c-a266-e63852327686 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.466266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559dac3-1739-4b8c-a266-e63852327686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.469359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 771/1543] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:15.484435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060d312-e2a4-4740-add2-9fadbe0d062d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:15.498272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4559dac3-1739-4b8c-a266-e63852327686 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.499812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4559dac3-1739-4b8c-a266-e63852327686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.610746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 773/1544] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.614522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0e4d-36c9-43ce-97a0-dd2bd795d9ac None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.614522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0e4d-36c9-43ce-97a0-dd2bd795d9ac None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.614522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf0e4d-36c9-43ce-97a0-dd2bd795d9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.636661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060d312-e2a4-4740-add2-9fadbe0d062d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.642896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-42cf0e4d-36c9-43ce-97a0-dd2bd795d9ac None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.644360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-42cf0e4d-36c9-43ce-97a0-dd2bd795d9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.653049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28039cc-7cf1-459f-99f2-f6b0093f830f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.655460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28039cc-7cf1-459f-99f2-f6b0093f830f admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.657113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28039cc-7cf1-459f-99f2-f6b0093f830f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.657477 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28039cc-7cf1-459f-99f2-f6b0093f830f admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.658350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28039cc-7cf1-459f-99f2-f6b0093f830f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.660019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-e28039cc-7cf1-459f-99f2-f6b0093f830f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:22:15.660019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 06 01:22:15.660019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: Aug 06 01:22:15.661509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28039cc-7cf1-459f-99f2-f6b0093f830f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.662859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 774/1545] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.664468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1577f-21b7-4d0f-a254-b7b372511c43 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.665922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1577f-21b7-4d0f-a254-b7b372511c43 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.666399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1577f-21b7-4d0f-a254-b7b372511c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.689114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-13f1577f-21b7-4d0f-a254-b7b372511c43 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.710394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-13f1577f-21b7-4d0f-a254-b7b372511c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.710394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 775/1546] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.710394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79ff4a-ecfb-43c6-8748-3d5879526b35 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.710394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79ff4a-ecfb-43c6-8748-3d5879526b35 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.710394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79ff4a-ecfb-43c6-8748-3d5879526b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.721348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2a79ff4a-ecfb-43c6-8748-3d5879526b35 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.722558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2a79ff4a-ecfb-43c6-8748-3d5879526b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.731661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 776/1547] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.732946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572dd63-4ffc-4f1f-b2e7-44375a950dac None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.733237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572dd63-4ffc-4f1f-b2e7-44375a950dac None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.733509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572dd63-4ffc-4f1f-b2e7-44375a950dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.755542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a572dd63-4ffc-4f1f-b2e7-44375a950dac None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.756994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a572dd63-4ffc-4f1f-b2e7-44375a950dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.765421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 777/1548] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.766613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca65cc-e8ca-4495-bee8-1e08b60dec32 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.766915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca65cc-e8ca-4495-bee8-1e08b60dec32 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.767273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca65cc-e8ca-4495-bee8-1e08b60dec32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.768006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 772/1549] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/users => generated 287 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:15.769169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d517290-0bf1-4ba7-8073-485291dfa9df None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.769526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d517290-0bf1-4ba7-8073-485291dfa9df None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.769868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d517290-0bf1-4ba7-8073-485291dfa9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.786722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-41ca65cc-e8ca-4495-bee8-1e08b60dec32 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.787951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-41ca65cc-e8ca-4495-bee8-1e08b60dec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.789165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7d517290-0bf1-4ba7-8073-485291dfa9df None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.791053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7d517290-0bf1-4ba7-8073-485291dfa9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.796382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 778/1550] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.797648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaf249-2398-4da4-85ce-aafa0bc870ac None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.797921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaf249-2398-4da4-85ce-aafa0bc870ac None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.798196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaf249-2398-4da4-85ce-aafa0bc870ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.803653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 773/1551] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.803653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba7ffd-4fbb-4b8e-8ca6-650faebb6c65 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.803653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba7ffd-4fbb-4b8e-8ca6-650faebb6c65 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.803653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba7ffd-4fbb-4b8e-8ca6-650faebb6c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.820729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ccbaf249-2398-4da4-85ce-aafa0bc870ac None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.822321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-feba7ffd-4fbb-4b8e-8ca6-650faebb6c65 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.822574 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ccbaf249-2398-4da4-85ce-aafa0bc870ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.823774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-feba7ffd-4fbb-4b8e-8ca6-650faebb6c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.830707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 779/1552] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.835890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 774/1553] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.836082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28808d6-3332-4db3-93ef-37ba4e97d46e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.836082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28808d6-3332-4db3-93ef-37ba4e97d46e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.836082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28808d6-3332-4db3-93ef-37ba4e97d46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.836925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39168dc6-b535-46d6-a6e4-db09ddd2e51c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.836925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39168dc6-b535-46d6-a6e4-db09ddd2e51c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.836925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39168dc6-b535-46d6-a6e4-db09ddd2e51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.853955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e28808d6-3332-4db3-93ef-37ba4e97d46e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.855173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e28808d6-3332-4db3-93ef-37ba4e97d46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.856812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-39168dc6-b535-46d6-a6e4-db09ddd2e51c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.857976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-39168dc6-b535-46d6-a6e4-db09ddd2e51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.863124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 780/1554] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.864023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28808d6-3332-4db3-93ef-37ba4e97d46e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:15.873345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 775/1555] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.873345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39168dc6-b535-46d6-a6e4-db09ddd2e51c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:15.876058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28808d6-3332-4db3-93ef-37ba4e97d46e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.877701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39168dc6-b535-46d6-a6e4-db09ddd2e51c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.880612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4a3d6-1558-42e8-8dd1-e485f0793c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.881222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4a3d6-1558-42e8-8dd1-e485f0793c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.881507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4a3d6-1558-42e8-8dd1-e485f0793c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.881784 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4a3d6-1558-42e8-8dd1-e485f0793c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.882238 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5210f-85e4-4f96-a5b6-51bd08fa70bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.882476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4a3d6-1558-42e8-8dd1-e485f0793c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.882833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5210f-85e4-4f96-a5b6-51bd08fa70bb admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.883117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5210f-85e4-4f96-a5b6-51bd08fa70bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.883987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5210f-85e4-4f96-a5b6-51bd08fa70bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.884680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5210f-85e4-4f96-a5b6-51bd08fa70bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:15.884784 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4a3d6-1558-42e8-8dd1-e485f0793c50 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.884987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5210f-85e4-4f96-a5b6-51bd08fa70bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.885992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-94d5210f-85e4-4f96-a5b6-51bd08fa70bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:22:15.885992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 06 01:22:15.885992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: Aug 06 01:22:15.886865 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5210f-85e4-4f96-a5b6-51bd08fa70bb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.894961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 781/1556] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:15.896244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842dfb6-ed58-44fe-803c-be857e240926 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.896537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842dfb6-ed58-44fe-803c-be857e240926 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.896815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842dfb6-ed58-44fe-803c-be857e240926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.901615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 776/1557] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:15.902434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5210f-85e4-4f96-a5b6-51bd08fa70bb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:15.912217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5210f-85e4-4f96-a5b6-51bd08fa70bb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.916812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b842dfb6-ed58-44fe-803c-be857e240926 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.916959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60b0f4-20ea-462c-aaf7-6b319021da7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.917447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60b0f4-20ea-462c-aaf7-6b319021da7d admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.917732 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60b0f4-20ea-462c-aaf7-6b319021da7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.918010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60b0f4-20ea-462c-aaf7-6b319021da7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.918123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b842dfb6-ed58-44fe-803c-be857e240926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.918700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60b0f4-20ea-462c-aaf7-6b319021da7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:15.919036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60b0f4-20ea-462c-aaf7-6b319021da7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.920678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60b0f4-20ea-462c-aaf7-6b319021da7d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.926120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 782/1558] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.926968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842dfb6-ed58-44fe-803c-be857e240926 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:15.933135 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 777/1559] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:15.934338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41d7a9-42d5-4c77-a22e-914e0896e701 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.934626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41d7a9-42d5-4c77-a22e-914e0896e701 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.934901 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41d7a9-42d5-4c77-a22e-914e0896e701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.944334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842dfb6-ed58-44fe-803c-be857e240926 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.949668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9fe0e-a264-4219-a5c3-c03771933196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.950309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9fe0e-a264-4219-a5c3-c03771933196 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.950584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9fe0e-a264-4219-a5c3-c03771933196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.951101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9fe0e-a264-4219-a5c3-c03771933196 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.951592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9fe0e-a264-4219-a5c3-c03771933196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:15.951931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9fe0e-a264-4219-a5c3-c03771933196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.953536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9fe0e-a264-4219-a5c3-c03771933196 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.956450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9e41d7a9-42d5-4c77-a22e-914e0896e701 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.957630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9e41d7a9-42d5-4c77-a22e-914e0896e701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.965440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 783/1560] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:15.969597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 778/1561] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.969780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9fe0e-a264-4219-a5c3-c03771933196 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:15.969919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456108d-c1e0-479c-b9bc-b54fe67ad138 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.969919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456108d-c1e0-479c-b9bc-b54fe67ad138 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.969919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456108d-c1e0-479c-b9bc-b54fe67ad138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.980730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9fe0e-a264-4219-a5c3-c03771933196 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:15.985274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75241922-ad2b-43d6-bd96-da152d6f623c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:15.985879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75241922-ad2b-43d6-bd96-da152d6f623c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:15.986162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75241922-ad2b-43d6-bd96-da152d6f623c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:15.986438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75241922-ad2b-43d6-bd96-da152d6f623c admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:15.987123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75241922-ad2b-43d6-bd96-da152d6f623c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:15.987427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75241922-ad2b-43d6-bd96-da152d6f623c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:15.987603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7456108d-c1e0-479c-b9bc-b54fe67ad138 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:15.989052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7456108d-c1e0-479c-b9bc-b54fe67ad138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:15.989220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75241922-ad2b-43d6-bd96-da152d6f623c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:15.996949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 779/1562] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:15.997707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456108d-c1e0-479c-b9bc-b54fe67ad138 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.001772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 784/1563] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:16.002700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75241922-ad2b-43d6-bd96-da152d6f623c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.010332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456108d-c1e0-479c-b9bc-b54fe67ad138 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.013504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75241922-ad2b-43d6-bd96-da152d6f623c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.015011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93a2e0-a334-4cdc-8fed-c3dfbcca4625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.015717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93a2e0-a334-4cdc-8fed-c3dfbcca4625 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.016033 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93a2e0-a334-4cdc-8fed-c3dfbcca4625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.016310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93a2e0-a334-4cdc-8fed-c3dfbcca4625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.017009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93a2e0-a334-4cdc-8fed-c3dfbcca4625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.018083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e0bc6-c629-452f-9465-b1b6b4044d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.018591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93a2e0-a334-4cdc-8fed-c3dfbcca4625 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.018707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0bc6-c629-452f-9465-b1b6b4044d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.018955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0bc6-c629-452f-9465-b1b6b4044d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.019234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0bc6-c629-452f-9465-b1b6b4044d12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.019932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e0bc6-c629-452f-9465-b1b6b4044d12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.020972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-902e0bc6-c629-452f-9465-b1b6b4044d12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:22:16.020972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 06 01:22:16.020972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: Aug 06 01:22:16.022159 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e0bc6-c629-452f-9465-b1b6b4044d12 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.029218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 780/1564] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:16.029990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93a2e0-a334-4cdc-8fed-c3dfbcca4625 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.040040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93a2e0-a334-4cdc-8fed-c3dfbcca4625 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.044796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8a142-d30f-4cd2-89f8-80230559f44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.046691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8a142-d30f-4cd2-89f8-80230559f44d admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.046691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8a142-d30f-4cd2-89f8-80230559f44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.046691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8a142-d30f-4cd2-89f8-80230559f44d admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.046691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8a142-d30f-4cd2-89f8-80230559f44d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:16.047050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8a142-d30f-4cd2-89f8-80230559f44d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.048502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8a142-d30f-4cd2-89f8-80230559f44d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.060825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 781/1565] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:16.061571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8a142-d30f-4cd2-89f8-80230559f44d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.070566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8a142-d30f-4cd2-89f8-80230559f44d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.075554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdefb8-bdf2-45ab-86f4-ea8194fbb660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.076156 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdefb8-bdf2-45ab-86f4-ea8194fbb660 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.076438 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdefb8-bdf2-45ab-86f4-ea8194fbb660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.076713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdefb8-bdf2-45ab-86f4-ea8194fbb660 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.077355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdefb8-bdf2-45ab-86f4-ea8194fbb660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.079186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cdefb8-bdf2-45ab-86f4-ea8194fbb660 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.095026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 785/1566] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/projects => generated 367 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:16.098453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e0bc6-c629-452f-9465-b1b6b4044d12 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.126039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:16.156517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e0bc6-c629-452f-9465-b1b6b4044d12 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.161207 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f3873-b059-42bf-95c8-bdefb5293771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.161825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f3873-b059-42bf-95c8-bdefb5293771 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.162131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f3873-b059-42bf-95c8-bdefb5293771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.162444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f3873-b059-42bf-95c8-bdefb5293771 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.163138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f3873-b059-42bf-95c8-bdefb5293771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:16.163441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f3873-b059-42bf-95c8-bdefb5293771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.165085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f3873-b059-42bf-95c8-bdefb5293771 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.176408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 786/1567] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:16.180191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f3873-b059-42bf-95c8-bdefb5293771 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.180288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 782/1568] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:16.181133 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdefb8-bdf2-45ab-86f4-ea8194fbb660 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.189565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f3873-b059-42bf-95c8-bdefb5293771 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.190417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:16.245150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdefb8-bdf2-45ab-86f4-ea8194fbb660 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.245785 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08bfa0-2e12-453a-8bca-b174590c77e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.246308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08bfa0-2e12-453a-8bca-b174590c77e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.246588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08bfa0-2e12-453a-8bca-b174590c77e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.246862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08bfa0-2e12-453a-8bca-b174590c77e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.247522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08bfa0-2e12-453a-8bca-b174590c77e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.249585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08bfa0-2e12-453a-8bca-b174590c77e0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.249978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3fa5-39ea-4ad7-b57d-4f5a7fecea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.250994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3fa5-39ea-4ad7-b57d-4f5a7fecea77 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.251384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3fa5-39ea-4ad7-b57d-4f5a7fecea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.251663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3fa5-39ea-4ad7-b57d-4f5a7fecea77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.252429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf3fa5-39ea-4ad7-b57d-4f5a7fecea77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:16.253270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf3fa5-39ea-4ad7-b57d-4f5a7fecea77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.254852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf3fa5-39ea-4ad7-b57d-4f5a7fecea77 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.266743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 783/1569] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:16.267537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3fa5-39ea-4ad7-b57d-4f5a7fecea77 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.277346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3fa5-39ea-4ad7-b57d-4f5a7fecea77 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.281925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457be543-52e6-4c21-ab97-2e36511f4d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.283496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457be543-52e6-4c21-ab97-2e36511f4d7c admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.283496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457be543-52e6-4c21-ab97-2e36511f4d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.283496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457be543-52e6-4c21-ab97-2e36511f4d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.283751 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457be543-52e6-4c21-ab97-2e36511f4d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.285607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457be543-52e6-4c21-ab97-2e36511f4d7c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.337940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 787/1570] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:16.338855 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08bfa0-2e12-453a-8bca-b174590c77e0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.383266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 784/1571] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/projects => generated 405 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:16.384349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457be543-52e6-4c21-ab97-2e36511f4d7c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.398543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08bfa0-2e12-453a-8bca-b174590c77e0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.448696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457be543-52e6-4c21-ab97-2e36511f4d7c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.449194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:16.453833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d335fb-57e8-41d4-976f-a70a0438f9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.459145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d335fb-57e8-41d4-976f-a70a0438f9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.459145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d335fb-57e8-41d4-976f-a70a0438f9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.459145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d335fb-57e8-41d4-976f-a70a0438f9d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.459145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d335fb-57e8-41d4-976f-a70a0438f9d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.459475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960171-07df-4da2-b470-9040686f08ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.459475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960171-07df-4da2-b470-9040686f08ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.459475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960171-07df-4da2-b470-9040686f08ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.459475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960171-07df-4da2-b470-9040686f08ee admin admin] PATH_INFO: `/v3/projects/d05c0fac722645dc9eee9074d3354781/users/762519187671476ba4738b2fbd9ec6f8/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.459917 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d335fb-57e8-41d4-976f-a70a0438f9d7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.472311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33960171-07df-4da2-b470-9040686f08ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05c0fac722645dc9eee9074d3354781, user_id=762519187671476ba4738b2fbd9ec6f8, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.473362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-33960171-07df-4da2-b470-9040686f08ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:22:16.473362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 06 01:22:16.473362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: Aug 06 01:22:16.475357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33960171-07df-4da2-b470-9040686f08ee admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.543187 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 785/1572] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/projects => generated 393 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:16.544116 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d335fb-57e8-41d4-976f-a70a0438f9d7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.564915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 788/1573] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:16 2022] PUT /identity/v3/projects/d05c0fac722645dc9eee9074d3354781/users/762519187671476ba4738b2fbd9ec6f8/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:16.570088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960171-07df-4da2-b470-9040686f08ee admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.621916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d335fb-57e8-41d4-976f-a70a0438f9d7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.626914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a72e5a-4d38-4862-a5da-90c15d772b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.627537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a72e5a-4d38-4862-a5da-90c15d772b6d admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.627841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a72e5a-4d38-4862-a5da-90c15d772b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.628120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a72e5a-4d38-4862-a5da-90c15d772b6d admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.628785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a72e5a-4d38-4862-a5da-90c15d772b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.630821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a72e5a-4d38-4862-a5da-90c15d772b6d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.631043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960171-07df-4da2-b470-9040686f08ee admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.636512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e208eca-964c-4788-938a-1c7dbd1df4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.637219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e208eca-964c-4788-938a-1c7dbd1df4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.637712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e208eca-964c-4788-938a-1c7dbd1df4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.638206 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e208eca-964c-4788-938a-1c7dbd1df4cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.639297 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e208eca-964c-4788-938a-1c7dbd1df4cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.641864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:16.642748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e208eca-964c-4788-938a-1c7dbd1df4cb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.713408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 786/1574] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/users => generated 284 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:16.714212 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a72e5a-4d38-4862-a5da-90c15d772b6d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.723565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a72e5a-4d38-4862-a5da-90c15d772b6d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.728863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dce38e-de42-4c5a-b5d6-3322c6824999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.729442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dce38e-de42-4c5a-b5d6-3322c6824999 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.729722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dce38e-de42-4c5a-b5d6-3322c6824999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.729997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dce38e-de42-4c5a-b5d6-3322c6824999 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.730647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dce38e-de42-4c5a-b5d6-3322c6824999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.732627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dce38e-de42-4c5a-b5d6-3322c6824999 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.741516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 789/1575] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:16.741516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e208eca-964c-4788-938a-1c7dbd1df4cb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.800410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e208eca-964c-4788-938a-1c7dbd1df4cb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.804933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4220348-6aee-4f09-9940-2b9a18b471bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.807170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4220348-6aee-4f09-9940-2b9a18b471bb admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.807170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4220348-6aee-4f09-9940-2b9a18b471bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.807170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4220348-6aee-4f09-9940-2b9a18b471bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.809231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4220348-6aee-4f09-9940-2b9a18b471bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.813475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4220348-6aee-4f09-9940-2b9a18b471bb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.848513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 787/1576] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:16.849324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dce38e-de42-4c5a-b5d6-3322c6824999 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.859527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dce38e-de42-4c5a-b5d6-3322c6824999 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:16.864194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeda58-7a2e-44e5-b1ba-de647d77ca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:16.864786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeda58-7a2e-44e5-b1ba-de647d77ca05 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:16.865062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeda58-7a2e-44e5-b1ba-de647d77ca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:16.865333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeda58-7a2e-44e5-b1ba-de647d77ca05 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:16.865977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeda58-7a2e-44e5-b1ba-de647d77ca05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:16.868583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeda58-7a2e-44e5-b1ba-de647d77ca05 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:16.960721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 790/1577] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/projects => generated 367 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:16.961665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4220348-6aee-4f09-9940-2b9a18b471bb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.983284 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 788/1578] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:16.983284 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeda58-7a2e-44e5-b1ba-de647d77ca05 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:16.992929 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeda58-7a2e-44e5-b1ba-de647d77ca05 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.012839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:17.026436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4220348-6aee-4f09-9940-2b9a18b471bb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.031208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa510b1-3263-42ba-9522-70c48a28878f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.031920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa510b1-3263-42ba-9522-70c48a28878f admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.032204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa510b1-3263-42ba-9522-70c48a28878f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.032478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa510b1-3263-42ba-9522-70c48a28878f admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.033140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa510b1-3263-42ba-9522-70c48a28878f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.035074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa510b1-3263-42ba-9522-70c48a28878f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.049564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d671f7d-14fa-46dc-9d0c-09b1fb3e100c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.053257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d671f7d-14fa-46dc-9d0c-09b1fb3e100c admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.053257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d671f7d-14fa-46dc-9d0c-09b1fb3e100c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.053257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d671f7d-14fa-46dc-9d0c-09b1fb3e100c admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.053257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d671f7d-14fa-46dc-9d0c-09b1fb3e100c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.053639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d671f7d-14fa-46dc-9d0c-09b1fb3e100c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.132921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 789/1579] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:17.134961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807217e5-1825-4d16-b6ce-601607456345 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.135305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807217e5-1825-4d16-b6ce-601607456345 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.135641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807217e5-1825-4d16-b6ce-601607456345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.136993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 791/1580] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:16 2022] POST /identity/v3/users => generated 303 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:17.137931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa510b1-3263-42ba-9522-70c48a28878f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.147796 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa510b1-3263-42ba-9522-70c48a28878f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.152592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75427b-d1fc-48eb-9fce-844da4c0f056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.153207 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75427b-d1fc-48eb-9fce-844da4c0f056 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.153492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75427b-d1fc-48eb-9fce-844da4c0f056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.153769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75427b-d1fc-48eb-9fce-844da4c0f056 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.154510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75427b-d1fc-48eb-9fce-844da4c0f056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.156138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75427b-d1fc-48eb-9fce-844da4c0f056 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.163698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-807217e5-1825-4d16-b6ce-601607456345 None None] MFA Rules not processed for user `762519187671476ba4738b2fbd9ec6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:17.165053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-807217e5-1825-4d16-b6ce-601607456345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:17.165933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 792/1581] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.166709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75427b-d1fc-48eb-9fce-844da4c0f056 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.176185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75427b-d1fc-48eb-9fce-844da4c0f056 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.181662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350465e9-1efc-4988-bf39-80787c6651df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.182253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350465e9-1efc-4988-bf39-80787c6651df admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.182534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350465e9-1efc-4988-bf39-80787c6651df admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.182808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350465e9-1efc-4988-bf39-80787c6651df admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.183458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350465e9-1efc-4988-bf39-80787c6651df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.185416 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350465e9-1efc-4988-bf39-80787c6651df admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.230382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 790/1582] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:17.231222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807217e5-1825-4d16-b6ce-601607456345 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.241312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807217e5-1825-4d16-b6ce-601607456345 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.246545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783431b-a6cb-4788-b517-797f40d0f1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.247145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783431b-a6cb-4788-b517-797f40d0f1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.247430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783431b-a6cb-4788-b517-797f40d0f1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.247712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783431b-a6cb-4788-b517-797f40d0f1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.248447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783431b-a6cb-4788-b517-797f40d0f1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.250033 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783431b-a6cb-4788-b517-797f40d0f1f5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.260335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 791/1583] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.261230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783431b-a6cb-4788-b517-797f40d0f1f5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.270642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783431b-a6cb-4788-b517-797f40d0f1f5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.270862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 793/1584] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:17 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:17.271866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350465e9-1efc-4988-bf39-80787c6651df admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.276143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72213e-5955-4d43-bf2e-fe894d956a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.276748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72213e-5955-4d43-bf2e-fe894d956a7c admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.277976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72213e-5955-4d43-bf2e-fe894d956a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.277976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72213e-5955-4d43-bf2e-fe894d956a7c admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.277976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72213e-5955-4d43-bf2e-fe894d956a7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.280087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72213e-5955-4d43-bf2e-fe894d956a7c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.282240 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350465e9-1efc-4988-bf39-80787c6651df admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.286761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f80eb-a4f5-4f62-96b0-0e900d65d778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.287360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f80eb-a4f5-4f62-96b0-0e900d65d778 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.287644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f80eb-a4f5-4f62-96b0-0e900d65d778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.287956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f80eb-a4f5-4f62-96b0-0e900d65d778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.288695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f80eb-a4f5-4f62-96b0-0e900d65d778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.290891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f80eb-a4f5-4f62-96b0-0e900d65d778 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.301463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 794/1585] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.302235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f80eb-a4f5-4f62-96b0-0e900d65d778 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.311888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f80eb-a4f5-4f62-96b0-0e900d65d778 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.316619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5461cb-c726-42cc-8bfe-43058fe00a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.317204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5461cb-c726-42cc-8bfe-43058fe00a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.317496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5461cb-c726-42cc-8bfe-43058fe00a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.317775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5461cb-c726-42cc-8bfe-43058fe00a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.318477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5461cb-c726-42cc-8bfe-43058fe00a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.320186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5461cb-c726-42cc-8bfe-43058fe00a11 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.329746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 795/1586] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.330507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5461cb-c726-42cc-8bfe-43058fe00a11 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.340638 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5461cb-c726-42cc-8bfe-43058fe00a11 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.345194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0affec-fd87-4f51-9d51-73a25c6e36a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.345749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0affec-fd87-4f51-9d51-73a25c6e36a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.346032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0affec-fd87-4f51-9d51-73a25c6e36a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.346307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0affec-fd87-4f51-9d51-73a25c6e36a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.346957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0affec-fd87-4f51-9d51-73a25c6e36a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.348513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0affec-fd87-4f51-9d51-73a25c6e36a5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.358518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 796/1587] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.359344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0affec-fd87-4f51-9d51-73a25c6e36a5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.362311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 792/1588] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:17 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:17.363152 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72213e-5955-4d43-bf2e-fe894d956a7c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.369047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0affec-fd87-4f51-9d51-73a25c6e36a5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.373620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a26fb-545f-4c37-9fd4-bddb6aaa568a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.374212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a26fb-545f-4c37-9fd4-bddb6aaa568a admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.374351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72213e-5955-4d43-bf2e-fe894d956a7c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.374496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a26fb-545f-4c37-9fd4-bddb6aaa568a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.374773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a26fb-545f-4c37-9fd4-bddb6aaa568a admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.375456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a26fb-545f-4c37-9fd4-bddb6aaa568a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.377065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a26fb-545f-4c37-9fd4-bddb6aaa568a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.379037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11c43a-d6a8-4769-a815-92533a791358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.379640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11c43a-d6a8-4769-a815-92533a791358 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.379965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11c43a-d6a8-4769-a815-92533a791358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.380248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11c43a-d6a8-4769-a815-92533a791358 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.380910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11c43a-d6a8-4769-a815-92533a791358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.382792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11c43a-d6a8-4769-a815-92533a791358 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.387359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 797/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.388243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a26fb-545f-4c37-9fd4-bddb6aaa568a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.389224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:INSERT=13,keystone:SELECT=87 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:17.397752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a26fb-545f-4c37-9fd4-bddb6aaa568a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.403083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94218c7e-7f7c-45c4-87b6-5d9c5ba93e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.403752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94218c7e-7f7c-45c4-87b6-5d9c5ba93e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.404069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94218c7e-7f7c-45c4-87b6-5d9c5ba93e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.404345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94218c7e-7f7c-45c4-87b6-5d9c5ba93e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.405043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94218c7e-7f7c-45c4-87b6-5d9c5ba93e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.406616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94218c7e-7f7c-45c4-87b6-5d9c5ba93e40 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.417895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 798/1590] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.423247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94218c7e-7f7c-45c4-87b6-5d9c5ba93e40 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.429234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94218c7e-7f7c-45c4-87b6-5d9c5ba93e40 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.433668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606621f9-5b66-44d5-9a7e-2ac375bf9167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.434256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606621f9-5b66-44d5-9a7e-2ac375bf9167 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.434536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606621f9-5b66-44d5-9a7e-2ac375bf9167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.434808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606621f9-5b66-44d5-9a7e-2ac375bf9167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.435474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606621f9-5b66-44d5-9a7e-2ac375bf9167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.437093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606621f9-5b66-44d5-9a7e-2ac375bf9167 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.446723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 799/1591] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.447478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606621f9-5b66-44d5-9a7e-2ac375bf9167 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.456502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606621f9-5b66-44d5-9a7e-2ac375bf9167 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.460985 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48e17d-f377-46b6-8ed1-73136ac68d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.462051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48e17d-f377-46b6-8ed1-73136ac68d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.462327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48e17d-f377-46b6-8ed1-73136ac68d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.462603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48e17d-f377-46b6-8ed1-73136ac68d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.463253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48e17d-f377-46b6-8ed1-73136ac68d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.464814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48e17d-f377-46b6-8ed1-73136ac68d2b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.474592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 800/1592] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.475339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48e17d-f377-46b6-8ed1-73136ac68d2b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.485636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48e17d-f377-46b6-8ed1-73136ac68d2b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.488092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 793/1593] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:17 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:17.488092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11c43a-d6a8-4769-a815-92533a791358 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.541997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51b542-ac08-4d61-8711-9165cc5f3e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.542611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51b542-ac08-4d61-8711-9165cc5f3e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.542894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51b542-ac08-4d61-8711-9165cc5f3e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.543168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51b542-ac08-4d61-8711-9165cc5f3e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.543891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51b542-ac08-4d61-8711-9165cc5f3e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.545524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51b542-ac08-4d61-8711-9165cc5f3e38 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.552746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11c43a-d6a8-4769-a815-92533a791358 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.555447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 801/1594] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.556299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51b542-ac08-4d61-8711-9165cc5f3e38 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.557793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94061eed-791c-4d27-af0e-0a240240fcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.558761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94061eed-791c-4d27-af0e-0a240240fcb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.559049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94061eed-791c-4d27-af0e-0a240240fcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.559329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94061eed-791c-4d27-af0e-0a240240fcb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.560068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94061eed-791c-4d27-af0e-0a240240fcb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.561664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94061eed-791c-4d27-af0e-0a240240fcb3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.566026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51b542-ac08-4d61-8711-9165cc5f3e38 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.571214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af54a4-4fce-480c-b419-efbe06efafb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.571819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af54a4-4fce-480c-b419-efbe06efafb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.575063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af54a4-4fce-480c-b419-efbe06efafb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.575208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 794/1595] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.575313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af54a4-4fce-480c-b419-efbe06efafb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.575313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af54a4-4fce-480c-b419-efbe06efafb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.575488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94061eed-791c-4d27-af0e-0a240240fcb3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.575582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af54a4-4fce-480c-b419-efbe06efafb0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.582693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94061eed-791c-4d27-af0e-0a240240fcb3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.588413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 802/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.588413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af54a4-4fce-480c-b419-efbe06efafb0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.591240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb13b5e-a4c7-4139-a9c4-74e9f30de6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.591952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb13b5e-a4c7-4139-a9c4-74e9f30de6f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.592235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb13b5e-a4c7-4139-a9c4-74e9f30de6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.592512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb13b5e-a4c7-4139-a9c4-74e9f30de6f6 admin admin] PATH_INFO: `/v3/projects/0741f73bbd304e68a37194eb13a897f6/users/d66a58cf4ac146f39d421e7844e1303a/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.597929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af54a4-4fce-480c-b419-efbe06efafb0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.603344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84ba37-313c-495d-bf3d-64992708e4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.604028 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84ba37-313c-495d-bf3d-64992708e4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.604322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84ba37-313c-495d-bf3d-64992708e4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.604601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84ba37-313c-495d-bf3d-64992708e4f0 admin admin] PATH_INFO: `/v3/projects/1ce28e8fffc1495cba5caca6092a3318/users/2d417104bf4040d4ae9ccb08097b5255/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.608251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb13b5e-a4c7-4139-a9c4-74e9f30de6f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0741f73bbd304e68a37194eb13a897f6, user_id=d66a58cf4ac146f39d421e7844e1303a, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.610965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb13b5e-a4c7-4139-a9c4-74e9f30de6f6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.620068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84ba37-313c-495d-bf3d-64992708e4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce28e8fffc1495cba5caca6092a3318, user_id=2d417104bf4040d4ae9ccb08097b5255, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.622749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84ba37-313c-495d-bf3d-64992708e4f0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.624972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:17.707189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 795/1597] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:17 2022] PUT /identity/v3/projects/0741f73bbd304e68a37194eb13a897f6/users/d66a58cf4ac146f39d421e7844e1303a/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:17.708041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb13b5e-a4c7-4139-a9c4-74e9f30de6f6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.719401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 803/1598] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:17 2022] PUT /identity/v3/projects/1ce28e8fffc1495cba5caca6092a3318/users/2d417104bf4040d4ae9ccb08097b5255/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:17.721934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84ba37-313c-495d-bf3d-64992708e4f0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.776488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb13b5e-a4c7-4139-a9c4-74e9f30de6f6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.781106 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c68c5-0cd4-4245-b76d-b6eecb5eca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.781720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c68c5-0cd4-4245-b76d-b6eecb5eca1b admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.782002 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c68c5-0cd4-4245-b76d-b6eecb5eca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.782284 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c68c5-0cd4-4245-b76d-b6eecb5eca1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.782961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84ba37-313c-495d-bf3d-64992708e4f0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.783092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c68c5-0cd4-4245-b76d-b6eecb5eca1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.784635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c68c5-0cd4-4245-b76d-b6eecb5eca1b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.787724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917b6c-0fde-4771-b52c-e41c1d9ae64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.788421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917b6c-0fde-4771-b52c-e41c1d9ae64f admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.788708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917b6c-0fde-4771-b52c-e41c1d9ae64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.788991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8917b6c-0fde-4771-b52c-e41c1d9ae64f admin admin] PATH_INFO: `/v3/projects/977a88ff5e02414cbb61184a82d3b850/users/5a2e467a14544f00b1dda2c0006386f0/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.798129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 796/1599] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:17.798129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c68c5-0cd4-4245-b76d-b6eecb5eca1b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.804859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8917b6c-0fde-4771-b52c-e41c1d9ae64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=977a88ff5e02414cbb61184a82d3b850, user_id=5a2e467a14544f00b1dda2c0006386f0, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.805013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c68c5-0cd4-4245-b76d-b6eecb5eca1b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.808179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8917b6c-0fde-4771-b52c-e41c1d9ae64f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.810180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ef758-d79e-4115-bb0e-ddde376e9a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.810851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ef758-d79e-4115-bb0e-ddde376e9a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.811205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ef758-d79e-4115-bb0e-ddde376e9a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.811486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ef758-d79e-4115-bb0e-ddde376e9a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.812216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ef758-d79e-4115-bb0e-ddde376e9a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.814590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ef758-d79e-4115-bb0e-ddde376e9a2a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:17.907048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 804/1600] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:17 2022] PUT /identity/v3/projects/977a88ff5e02414cbb61184a82d3b850/users/5a2e467a14544f00b1dda2c0006386f0/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:17.907947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917b6c-0fde-4771-b52c-e41c1d9ae64f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.936151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 797/1601] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:17 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:17.936965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ef758-d79e-4115-bb0e-ddde376e9a2a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:17.947534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ef758-d79e-4115-bb0e-ddde376e9a2a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.972470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8917b6c-0fde-4771-b52c-e41c1d9ae64f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:17.977213 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75110466-a660-4cb7-9155-b52789eedcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:17.978158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75110466-a660-4cb7-9155-b52789eedcdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:17.978571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75110466-a660-4cb7-9155-b52789eedcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:17.978962 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75110466-a660-4cb7-9155-b52789eedcdb admin admin] PATH_INFO: `/v3/projects/1b517eec95d74c09a3a7e21716b364c9/users/cf36db92027f4d21be64dc085f63aac2/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:17.981631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:17.994688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75110466-a660-4cb7-9155-b52789eedcdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b517eec95d74c09a3a7e21716b364c9, user_id=cf36db92027f4d21be64dc085f63aac2, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:17.997444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75110466-a660-4cb7-9155-b52789eedcdb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.004989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f664b-653b-4756-a405-02076924c39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.005694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f664b-653b-4756-a405-02076924c39f admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.005990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f664b-653b-4756-a405-02076924c39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.006261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f664b-653b-4756-a405-02076924c39f admin admin] PATH_INFO: `/v3/projects/fd509ff0932d40beb6ef2712e99a5fca/users/1c96cc61c2b84b3b81ed6c6738f9773e/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.021256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f664b-653b-4756-a405-02076924c39f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd509ff0932d40beb6ef2712e99a5fca, user_id=1c96cc61c2b84b3b81ed6c6738f9773e, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:18.026465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f664b-653b-4756-a405-02076924c39f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.101745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 805/1602] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:17 2022] PUT /identity/v3/projects/1b517eec95d74c09a3a7e21716b364c9/users/cf36db92027f4d21be64dc085f63aac2/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:18.103424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75110466-a660-4cb7-9155-b52789eedcdb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.113781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 798/1603] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:17 2022] PUT /identity/v3/projects/fd509ff0932d40beb6ef2712e99a5fca/users/1c96cc61c2b84b3b81ed6c6738f9773e/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:18.117405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbefa68-cb46-406e-8715-70b9191b5a2e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.117405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbefa68-cb46-406e-8715-70b9191b5a2e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.117405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbefa68-cb46-406e-8715-70b9191b5a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.142571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2bbefa68-cb46-406e-8715-70b9191b5a2e None None] MFA Rules not processed for user `d66a58cf4ac146f39d421e7844e1303a`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:18.143802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2bbefa68-cb46-406e-8715-70b9191b5a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:18.168441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75110466-a660-4cb7-9155-b52789eedcdb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:18.171941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:18.208203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 799/1604] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:18.212870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ae21-f235-4506-b437-125e5885705c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.214719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ae21-f235-4506-b437-125e5885705c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.214719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ae21-f235-4506-b437-125e5885705c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.225031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa826cf2-8cca-4824-8975-7aabc963bc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.225677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa826cf2-8cca-4824-8975-7aabc963bc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.225964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa826cf2-8cca-4824-8975-7aabc963bc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.226241 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa826cf2-8cca-4824-8975-7aabc963bc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.226975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa826cf2-8cca-4824-8975-7aabc963bc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:18.229610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa826cf2-8cca-4824-8975-7aabc963bc8c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.236361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-af79ae21-f235-4506-b437-125e5885705c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:18.237583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-af79ae21-f235-4506-b437-125e5885705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:18.243988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 806/1605] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:18.243988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa826cf2-8cca-4824-8975-7aabc963bc8c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.245453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 800/1606] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:18.247799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462573d-1008-4860-88e2-3e1910f2c3a5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.248375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462573d-1008-4860-88e2-3e1910f2c3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.248375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462573d-1008-4860-88e2-3e1910f2c3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.253187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa826cf2-8cca-4824-8975-7aabc963bc8c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:18.258008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d2d1a-1706-4d0d-bae5-f8ec4d2cbd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.259204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d2d1a-1706-4d0d-bae5-f8ec4d2cbd60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.259490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d2d1a-1706-4d0d-bae5-f8ec4d2cbd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.259768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d2d1a-1706-4d0d-bae5-f8ec4d2cbd60 admin admin] PATH_INFO: `/v3/projects/ced29d168e064480a2d3619a7d8fca88/users/90f54ffc10a447d0a873f3e7c6ab2647/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.276251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d2d1a-1706-4d0d-bae5-f8ec4d2cbd60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ced29d168e064480a2d3619a7d8fca88, user_id=90f54ffc10a447d0a873f3e7c6ab2647, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:18.276541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f462573d-1008-4860-88e2-3e1910f2c3a5 None None] MFA Rules not processed for user `5a2e467a14544f00b1dda2c0006386f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:18.277764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f462573d-1008-4860-88e2-3e1910f2c3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:18.279101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d2d1a-1706-4d0d-bae5-f8ec4d2cbd60 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.340569 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 801/1607] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:18.341402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462573d-1008-4860-88e2-3e1910f2c3a5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.354248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 807/1608] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:18 2022] PUT /identity/v3/projects/ced29d168e064480a2d3619a7d8fca88/users/90f54ffc10a447d0a873f3e7c6ab2647/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:18.354248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46be7b-a4d2-41e3-a495-0f0f21c7a2ca None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.354248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46be7b-a4d2-41e3-a495-0f0f21c7a2ca None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.354248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46be7b-a4d2-41e3-a495-0f0f21c7a2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.355803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462573d-1008-4860-88e2-3e1910f2c3a5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:18.371873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-db46be7b-a4d2-41e3-a495-0f0f21c7a2ca None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:18.374033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-db46be7b-a4d2-41e3-a495-0f0f21c7a2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:18.388135 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:18.414777 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796af65-6032-4165-99e5-662fee4f7a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.415405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796af65-6032-4165-99e5-662fee4f7a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.415692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796af65-6032-4165-99e5-662fee4f7a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.416005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796af65-6032-4165-99e5-662fee4f7a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.416711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796af65-6032-4165-99e5-662fee4f7a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:18.418318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796af65-6032-4165-99e5-662fee4f7a1e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.432231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 802/1609] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:18.432231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce173d71-bc01-4492-854f-969f95d67dff None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.432231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce173d71-bc01-4492-854f-969f95d67dff None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.432231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce173d71-bc01-4492-854f-969f95d67dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.436981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 808/1610] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:18.438021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46be7b-a4d2-41e3-a495-0f0f21c7a2ca None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.450809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ce173d71-bc01-4492-854f-969f95d67dff None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:18.452053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ce173d71-bc01-4492-854f-969f95d67dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:18.459710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 803/1611] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:18.464254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce173d71-bc01-4492-854f-969f95d67dff None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.473421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce173d71-bc01-4492-854f-969f95d67dff None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:18.479883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffc4d3-2d0b-49fb-ba07-65ed77e9e75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.480546 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffc4d3-2d0b-49fb-ba07-65ed77e9e75a admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.480835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffc4d3-2d0b-49fb-ba07-65ed77e9e75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.481115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffc4d3-2d0b-49fb-ba07-65ed77e9e75a admin admin] PATH_INFO: `/v3/projects/0f149f11ee014e6baf15c98444f91075/users/702b55c0640047af91e739ce22d48098/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.495995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffc4d3-2d0b-49fb-ba07-65ed77e9e75a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f149f11ee014e6baf15c98444f91075, user_id=702b55c0640047af91e739ce22d48098, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:18.498650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffc4d3-2d0b-49fb-ba07-65ed77e9e75a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.504507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46be7b-a4d2-41e3-a495-0f0f21c7a2ca None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:18.508270 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc49d51-b86a-494f-b3b8-4b1b86875747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.508927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc49d51-b86a-494f-b3b8-4b1b86875747 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.509219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc49d51-b86a-494f-b3b8-4b1b86875747 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.509497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc49d51-b86a-494f-b3b8-4b1b86875747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.541257 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:18.565164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc49d51-b86a-494f-b3b8-4b1b86875747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:18.566591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc49d51-b86a-494f-b3b8-4b1b86875747 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.567216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 804/1612] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:18 2022] PUT /identity/v3/projects/0f149f11ee014e6baf15c98444f91075/users/702b55c0640047af91e739ce22d48098/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:18.568084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffc4d3-2d0b-49fb-ba07-65ed77e9e75a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.574906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-45ffc4d3-2d0b-49fb-ba07-65ed77e9e75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:18.575712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 809/1613] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:18.580072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77849b07-38e1-416b-b952-925d1abd0587 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.580072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77849b07-38e1-416b-b952-925d1abd0587 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.580072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77849b07-38e1-416b-b952-925d1abd0587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.597066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-77849b07-38e1-416b-b952-925d1abd0587 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:18.598341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-77849b07-38e1-416b-b952-925d1abd0587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:18.646219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffc4d3-2d0b-49fb-ba07-65ed77e9e75a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:18.650707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d0970-8cee-4110-9724-c6eca2cbdfcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.651402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d0970-8cee-4110-9724-c6eca2cbdfcd service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.651792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d0970-8cee-4110-9724-c6eca2cbdfcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.652164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d0970-8cee-4110-9724-c6eca2cbdfcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.663362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 810/1614] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:18.663362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90de1d-f353-46e2-8a5f-db1cb68fffb8 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.663362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90de1d-f353-46e2-8a5f-db1cb68fffb8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.663653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90de1d-f353-46e2-8a5f-db1cb68fffb8 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.665454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 811/1615] 10.222.0.25 () {54 vars in 962 bytes} [Sat Aug 6 01:22:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:18.668617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90de1d-f353-46e2-8a5f-db1cb68fffb8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.671138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d0970-8cee-4110-9724-c6eca2cbdfcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:18.672902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d0970-8cee-4110-9724-c6eca2cbdfcd service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.683681 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 805/1616] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:18.684571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d0970-8cee-4110-9724-c6eca2cbdfcd service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.687590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90de1d-f353-46e2-8a5f-db1cb68fffb8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:18.692799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d0970-8cee-4110-9724-c6eca2cbdfcd service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:18.696684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e68a8b-0b3c-4317-9cc1-8427b06e3722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.697472 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d4b7e-cf0b-42e0-9a5f-bad33b2d8c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.697618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e68a8b-0b3c-4317-9cc1-8427b06e3722 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.697910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e68a8b-0b3c-4317-9cc1-8427b06e3722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.698222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e68a8b-0b3c-4317-9cc1-8427b06e3722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.699076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d4b7e-cf0b-42e0-9a5f-bad33b2d8c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.699573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d4b7e-cf0b-42e0-9a5f-bad33b2d8c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.700091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d4b7e-cf0b-42e0-9a5f-bad33b2d8c12 admin admin] PATH_INFO: `/v3/projects/d066c13020cc45b6a43ce4cb60461d50/users/34277a0b4681446bb3920420edd2b5d8/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.708399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e68a8b-0b3c-4317-9cc1-8427b06e3722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:18.709891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e68a8b-0b3c-4317-9cc1-8427b06e3722 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.720812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d4b7e-cf0b-42e0-9a5f-bad33b2d8c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d066c13020cc45b6a43ce4cb60461d50, user_id=34277a0b4681446bb3920420edd2b5d8, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:18.723865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d4b7e-cf0b-42e0-9a5f-bad33b2d8c12 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.725092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 806/1617] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:18.726008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e68a8b-0b3c-4317-9cc1-8427b06e3722 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.735011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e68a8b-0b3c-4317-9cc1-8427b06e3722 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:18.738998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723a218-ca81-4014-ab92-96eb33b33ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.739701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723a218-ca81-4014-ab92-96eb33b33ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.742720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723a218-ca81-4014-ab92-96eb33b33ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.742720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723a218-ca81-4014-ab92-96eb33b33ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.750490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723a218-ca81-4014-ab92-96eb33b33ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:18.752087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723a218-ca81-4014-ab92-96eb33b33ed2 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:18.761891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 807/1618] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:18.763188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74a5a-4885-4943-9f2c-daa24099a8f8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.763550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74a5a-4885-4943-9f2c-daa24099a8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.763938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74a5a-4885-4943-9f2c-daa24099a8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.794812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a8f74a5a-4885-4943-9f2c-daa24099a8f8 None None] MFA Rules not processed for user `702b55c0640047af91e739ce22d48098`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:18.796165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a8f74a5a-4885-4943-9f2c-daa24099a8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:18.802836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 812/1619] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:18 2022] PUT /identity/v3/projects/d066c13020cc45b6a43ce4cb60461d50/users/34277a0b4681446bb3920420edd2b5d8/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:18.805795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da02ea6-2a8d-4e7a-adf7-e13a65e33ad7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.805795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da02ea6-2a8d-4e7a-adf7-e13a65e33ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.805795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da02ea6-2a8d-4e7a-adf7-e13a65e33ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.861806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5da02ea6-2a8d-4e7a-adf7-e13a65e33ad7 None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:18.863189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5da02ea6-2a8d-4e7a-adf7-e13a65e33ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:18.891852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 808/1620] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:18.891852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74a5a-4885-4943-9f2c-daa24099a8f8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.901654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:18.952623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 813/1621] 10.222.0.25 () {58 vars in 1059 bytes} [Sat Aug 6 01:22:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:18.954555 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da02ea6-2a8d-4e7a-adf7-e13a65e33ad7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:18.975847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74a5a-4885-4943-9f2c-daa24099a8f8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:18.982550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bba30-8fb5-46bd-9e5a-10f4232b95ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:18.983024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bba30-8fb5-46bd-9e5a-10f4232b95ff service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:18.983417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bba30-8fb5-46bd-9e5a-10f4232b95ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:18.983694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bba30-8fb5-46bd-9e5a-10f4232b95ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:18.999107 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:19.054354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da02ea6-2a8d-4e7a-adf7-e13a65e33ad7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:19.061590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf2763-7d14-4be3-8495-d24a38a134e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:19.062313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf2763-7d14-4be3-8495-d24a38a134e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.063090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf2763-7d14-4be3-8495-d24a38a134e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.063383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf2763-7d14-4be3-8495-d24a38a134e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.064203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf2763-7d14-4be3-8495-d24a38a134e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:19.065792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bba30-8fb5-46bd-9e5a-10f4232b95ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:19.065923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf2763-7d14-4be3-8495-d24a38a134e4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:19.067695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bba30-8fb5-46bd-9e5a-10f4232b95ff service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:19.082177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 809/1622] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:19.082278 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 814/1623] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:19.089824 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bba30-8fb5-46bd-9e5a-10f4232b95ff service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:19.090109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf2763-7d14-4be3-8495-d24a38a134e4 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:19.112368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bba30-8fb5-46bd-9e5a-10f4232b95ff service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:19.119350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0c332-49a4-4a4c-b8b4-4b840e69f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:19.120200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0c332-49a4-4a4c-b8b4-4b840e69f26e admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.120487 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0c332-49a4-4a4c-b8b4-4b840e69f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.120765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0c332-49a4-4a4c-b8b4-4b840e69f26e admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.122122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0c332-49a4-4a4c-b8b4-4b840e69f26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:19.124836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0c332-49a4-4a4c-b8b4-4b840e69f26e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:19.126555 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf2763-7d14-4be3-8495-d24a38a134e4 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:19.139334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b84ec-76e5-426d-8fdd-9f4ab228b764 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:19.139334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b84ec-76e5-426d-8fdd-9f4ab228b764 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.139334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b84ec-76e5-426d-8fdd-9f4ab228b764 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.139334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b84ec-76e5-426d-8fdd-9f4ab228b764 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.210544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b84ec-76e5-426d-8fdd-9f4ab228b764 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:19.212848 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b84ec-76e5-426d-8fdd-9f4ab228b764 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:19.223882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 815/1624] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:19.225440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b84ec-76e5-426d-8fdd-9f4ab228b764 service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:19.235932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b84ec-76e5-426d-8fdd-9f4ab228b764 service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:19.241022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771abe0-46be-4ca1-a990-cf6896d3dbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:19.241894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771abe0-46be-4ca1-a990-cf6896d3dbac admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.242183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771abe0-46be-4ca1-a990-cf6896d3dbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.245906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771abe0-46be-4ca1-a990-cf6896d3dbac admin admin] PATH_INFO: `/v3/projects/d066c13020cc45b6a43ce4cb60461d50/users/34277a0b4681446bb3920420edd2b5d8/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.248317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771abe0-46be-4ca1-a990-cf6896d3dbac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d066c13020cc45b6a43ce4cb60461d50, user_id=34277a0b4681446bb3920420edd2b5d8, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:19.257124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771abe0-46be-4ca1-a990-cf6896d3dbac admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:19.270261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 816/1625] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:19 2022] PUT /identity/v3/projects/d066c13020cc45b6a43ce4cb60461d50/users/34277a0b4681446bb3920420edd2b5d8/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:19.281040 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bf22-9d74-465d-a152-40e36b0559f6 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.282826 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bf22-9d74-465d-a152-40e36b0559f6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.282826 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bf22-9d74-465d-a152-40e36b0559f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.286840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 810/1626] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:19 2022] POST /identity/v3/projects => generated 369 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:19.287716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0c332-49a4-4a4c-b8b4-4b840e69f26e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:19.325073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8a14bf22-9d74-465d-a152-40e36b0559f6 None None] MFA Rules not processed for user `34277a0b4681446bb3920420edd2b5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:19.326347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8a14bf22-9d74-465d-a152-40e36b0559f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:19.387582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0c332-49a4-4a4c-b8b4-4b840e69f26e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:19.392997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9e0e9-785c-4045-82a6-bcf017248892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:19.393549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9e0e9-785c-4045-82a6-bcf017248892 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.393802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9e0e9-785c-4045-82a6-bcf017248892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.394141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9e0e9-785c-4045-82a6-bcf017248892 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.394877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9e0e9-785c-4045-82a6-bcf017248892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:19.397004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9e0e9-785c-4045-82a6-bcf017248892 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:19.448595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 817/1627] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:19.463224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14bf22-9d74-465d-a152-40e36b0559f6 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:19.503221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 811/1628] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:19 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:19.505275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9e0e9-785c-4045-82a6-bcf017248892 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:19.522633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9e0e9-785c-4045-82a6-bcf017248892 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:19.540927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8a7f-ad37-44eb-b9e0-a0c223fe2412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:19.540927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8a7f-ad37-44eb-b9e0-a0c223fe2412 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.540927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8a7f-ad37-44eb-b9e0-a0c223fe2412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.540927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8a7f-ad37-44eb-b9e0-a0c223fe2412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.541580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb8a7f-ad37-44eb-b9e0-a0c223fe2412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:19.542285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb8a7f-ad37-44eb-b9e0-a0c223fe2412 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:19.550281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:19.574782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 812/1629] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:19.575974 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8a7f-ad37-44eb-b9e0-a0c223fe2412 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:19.596960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8a7f-ad37-44eb-b9e0-a0c223fe2412 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:19.625418 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14bf22-9d74-465d-a152-40e36b0559f6 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:19.625418 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca24da6-7629-46ae-b505-3f28745be57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:19.625418 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca24da6-7629-46ae-b505-3f28745be57a service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.625418 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca24da6-7629-46ae-b505-3f28745be57a service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.625724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca24da6-7629-46ae-b505-3f28745be57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.636939 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca24da6-7629-46ae-b505-3f28745be57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:19.638878 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca24da6-7629-46ae-b505-3f28745be57a service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:19.643396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d851b-178f-40d1-be96-f824ec13acd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:19.646003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d851b-178f-40d1-be96-f824ec13acd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.648007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d851b-178f-40d1-be96-f824ec13acd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.654458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d851b-178f-40d1-be96-f824ec13acd6 admin admin] PATH_INFO: `/v3/projects/bfb77c7235f74ccfb409cbe482c564ec/users/f4aecb224d9c46bd9a9bf2a7cfcd4078/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.668747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:19.673509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 818/1630] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:22:19.689850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d851b-178f-40d1-be96-f824ec13acd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb77c7235f74ccfb409cbe482c564ec, user_id=f4aecb224d9c46bd9a9bf2a7cfcd4078, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:19.692572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d851b-178f-40d1-be96-f824ec13acd6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:19.803275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 813/1631] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:19 2022] PUT /identity/v3/projects/bfb77c7235f74ccfb409cbe482c564ec/users/f4aecb224d9c46bd9a9bf2a7cfcd4078/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:19.807727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca24da6-7629-46ae-b505-3f28745be57a service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:19.823147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca24da6-7629-46ae-b505-3f28745be57a service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:19.949611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83393ce4-dd52-43dc-bc00-5e6622b78212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:19.949611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83393ce4-dd52-43dc-bc00-5e6622b78212 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.949611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83393ce4-dd52-43dc-bc00-5e6622b78212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.949611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83393ce4-dd52-43dc-bc00-5e6622b78212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.950260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83393ce4-dd52-43dc-bc00-5e6622b78212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:19.952453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83393ce4-dd52-43dc-bc00-5e6622b78212 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:19.967449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d851b-178f-40d1-be96-f824ec13acd6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:19.969604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 819/1632] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:19.971058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83393ce4-dd52-43dc-bc00-5e6622b78212 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:19.986044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83393ce4-dd52-43dc-bc00-5e6622b78212 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:19.996570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ddf93-a0fd-4726-9429-b626b9f76b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:19.996570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ddf93-a0fd-4726-9429-b626b9f76b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:19.996570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ddf93-a0fd-4726-9429-b626b9f76b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:19.996570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ddf93-a0fd-4726-9429-b626b9f76b1c admin admin] PATH_INFO: `/v3/projects/bfb77c7235f74ccfb409cbe482c564ec/users/f4aecb224d9c46bd9a9bf2a7cfcd4078/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:19.998688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ddf93-a0fd-4726-9429-b626b9f76b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb77c7235f74ccfb409cbe482c564ec, user_id=f4aecb224d9c46bd9a9bf2a7cfcd4078, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:20.019988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ddf93-a0fd-4726-9429-b626b9f76b1c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:20.050682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 820/1633] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:19 2022] PUT /identity/v3/projects/bfb77c7235f74ccfb409cbe482c564ec/users/f4aecb224d9c46bd9a9bf2a7cfcd4078/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:20.060917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b489a55-724e-4b80-b520-e97328c26e69 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.061324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b489a55-724e-4b80-b520-e97328c26e69 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.061681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b489a55-724e-4b80-b520-e97328c26e69 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.063381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 821/1634] 10.222.0.25 () {54 vars in 962 bytes} [Sat Aug 6 01:22:20 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:20.064528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c214f9a-4142-4215-9459-fd415e5291b4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.064877 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c214f9a-4142-4215-9459-fd415e5291b4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.065281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c214f9a-4142-4215-9459-fd415e5291b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.070212 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d851b-178f-40d1-be96-f824ec13acd6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:20.099920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2c214f9a-4142-4215-9459-fd415e5291b4 None None] MFA Rules not processed for user `f4aecb224d9c46bd9a9bf2a7cfcd4078`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:20.101336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2c214f9a-4142-4215-9459-fd415e5291b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:20.210963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660033d-3bc7-4310-84fd-716739fdb826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:20.212172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660033d-3bc7-4310-84fd-716739fdb826 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.216103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660033d-3bc7-4310-84fd-716739fdb826 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.217458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660033d-3bc7-4310-84fd-716739fdb826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.228367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 822/1635] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:20.230174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f632f6-017d-4182-bc84-c72cd450aad2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.230519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f632f6-017d-4182-bc84-c72cd450aad2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.230867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f632f6-017d-4182-bc84-c72cd450aad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.255203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-37f632f6-017d-4182-bc84-c72cd450aad2 None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:20.256541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-37f632f6-017d-4182-bc84-c72cd450aad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:20.318691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:20.318994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660033d-3bc7-4310-84fd-716739fdb826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:20.320472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660033d-3bc7-4310-84fd-716739fdb826 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:20.335854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 814/1636] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:22:20.340331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660033d-3bc7-4310-84fd-716739fdb826 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:20.365069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 823/1637] 10.222.0.25 () {58 vars in 1059 bytes} [Sat Aug 6 01:22:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:20.368449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660033d-3bc7-4310-84fd-716739fdb826 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:20.372869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e601990-3422-4f84-a365-2be723c80a30 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.373272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e601990-3422-4f84-a365-2be723c80a30 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.373709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e601990-3422-4f84-a365-2be723c80a30 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.375640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 824/1638] 10.222.0.25 () {54 vars in 912 bytes} [Sat Aug 6 01:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:20.378602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f99c7-6a32-4169-8888-38374facb5df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:20.379491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f99c7-6a32-4169-8888-38374facb5df service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.380009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f99c7-6a32-4169-8888-38374facb5df service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.380485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f99c7-6a32-4169-8888-38374facb5df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.382232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62913518-f013-42e1-bedb-72759613c686 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.383090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62913518-f013-42e1-bedb-72759613c686 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.384020 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62913518-f013-42e1-bedb-72759613c686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.414114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f99c7-6a32-4169-8888-38374facb5df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:20.419634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f99c7-6a32-4169-8888-38374facb5df service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:20.457836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-62913518-f013-42e1-bedb-72759613c686 None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:20.459084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-62913518-f013-42e1-bedb-72759613c686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:20.462544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 815/1639] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:20.472091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 825/1640] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:22:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:20.660045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f99c7-6a32-4169-8888-38374facb5df service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:20.669105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f99c7-6a32-4169-8888-38374facb5df service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:20.672819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9a0ed-b0a3-410d-8be8-888be6a51e13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:20.673424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9a0ed-b0a3-410d-8be8-888be6a51e13 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.673702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9a0ed-b0a3-410d-8be8-888be6a51e13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.673972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9a0ed-b0a3-410d-8be8-888be6a51e13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.690425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9a0ed-b0a3-410d-8be8-888be6a51e13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:20.691991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9a0ed-b0a3-410d-8be8-888be6a51e13 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:20.705803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 816/1641] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:20.810360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a0fe7-c976-49a8-b44c-c505cbce1d45 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.810907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a0fe7-c976-49a8-b44c-c505cbce1d45 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.811179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a0fe7-c976-49a8-b44c-c505cbce1d45 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.812718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 826/1642] 10.222.0.25 () {54 vars in 912 bytes} [Sat Aug 6 01:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:20.823665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eff666-e8df-4e3d-b283-0b92c45cff46 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.823665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eff666-e8df-4e3d-b283-0b92c45cff46 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.823665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eff666-e8df-4e3d-b283-0b92c45cff46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:20.890462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-33eff666-e8df-4e3d-b283-0b92c45cff46 None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:20.892113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-33eff666-e8df-4e3d-b283-0b92c45cff46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:20.912898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 817/1643] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:22:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:20.948077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec771-51c5-4a24-9529-0d77c5800d74 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:20.949025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec771-51c5-4a24-9529-0d77c5800d74 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:20.949785 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec771-51c5-4a24-9529-0d77c5800d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:21.037569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-399ec771-51c5-4a24-9529-0d77c5800d74 None None] MFA Rules not processed for user `cf36db92027f4d21be64dc085f63aac2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:21.039146 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-399ec771-51c5-4a24-9529-0d77c5800d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:21.075228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eff666-e8df-4e3d-b283-0b92c45cff46 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:21.101880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eff666-e8df-4e3d-b283-0b92c45cff46 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:21.107566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c783f-e208-44dd-aabb-21043cb992b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:21.114212 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c783f-e208-44dd-aabb-21043cb992b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:21.114212 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c783f-e208-44dd-aabb-21043cb992b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:21.114212 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c783f-e208-44dd-aabb-21043cb992b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:21.137574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c783f-e208-44dd-aabb-21043cb992b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:21.140673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c783f-e208-44dd-aabb-21043cb992b1 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:21.166386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 818/1644] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:21.184365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 827/1645] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:21.189101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c783f-e208-44dd-aabb-21043cb992b1 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:21.215659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:21.346116 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c783f-e208-44dd-aabb-21043cb992b1 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:21.365110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f782b-5182-459e-a485-2241afb40ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:21.365110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f782b-5182-459e-a485-2241afb40ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:21.366607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ec771-51c5-4a24-9529-0d77c5800d74 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:21.374742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f782b-5182-459e-a485-2241afb40ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:21.377450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f782b-5182-459e-a485-2241afb40ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:21.387428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f782b-5182-459e-a485-2241afb40ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:21.390396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f782b-5182-459e-a485-2241afb40ae1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:21.391764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ec771-51c5-4a24-9529-0d77c5800d74 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:21.623434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:21.623434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:21.623434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:21.623434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] PATH_INFO: `/v3/users/5a2e467a14544f00b1dda2c0006386f0` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:21.627031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2e467a14544f00b1dda2c0006386f0)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:21.627031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: WARNING py.warnings [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:22:21.627031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: warnings.warn(msg) Aug 06 01:22:21.627031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: Aug 06 01:22:21.627031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:21.628057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 819/1646] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:21 2022] POST /identity/v3/projects => generated 403 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:21.628057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f782b-5182-459e-a485-2241afb40ae1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:21.714790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f782b-5182-459e-a485-2241afb40ae1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:21.720288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046de32-4cb8-465e-8ef5-3d5ce52952de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:21.721623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046de32-4cb8-465e-8ef5-3d5ce52952de admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:21.721812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046de32-4cb8-465e-8ef5-3d5ce52952de admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:21.722275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046de32-4cb8-465e-8ef5-3d5ce52952de admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:21.722996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046de32-4cb8-465e-8ef5-3d5ce52952de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:21.725125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046de32-4cb8-465e-8ef5-3d5ce52952de admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:21.739909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2e467a14544f00b1dda2c0006386f0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:21.754421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2e467a14544f00b1dda2c0006386f0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:21.831038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 820/1647] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:21 2022] POST /identity/v3/users => generated 308 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:21.834563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046de32-4cb8-465e-8ef5-3d5ce52952de admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:21.842160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2e467a14544f00b1dda2c0006386f0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:21.842893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2e467a14544f00b1dda2c0006386f0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:21.843588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-63a86db8-4e7a-414a-9634-f946e0d7b944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2e467a14544f00b1dda2c0006386f0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:21.851837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046de32-4cb8-465e-8ef5-3d5ce52952de admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:21.859159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254dabb-3f2c-4f2e-a4ef-411362055107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:21.859972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254dabb-3f2c-4f2e-a4ef-411362055107 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:21.860420 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254dabb-3f2c-4f2e-a4ef-411362055107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:21.861129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254dabb-3f2c-4f2e-a4ef-411362055107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:21.862986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254dabb-3f2c-4f2e-a4ef-411362055107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:21.866132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254dabb-3f2c-4f2e-a4ef-411362055107 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:21.889535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 821/1648] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:21.891138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 828/1649] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:21 2022] DELETE /identity/v3/users/5a2e467a14544f00b1dda2c0006386f0 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:21.895843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6910f27-704d-4d7c-b4a1-1683a417ff43 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:21.896147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6910f27-704d-4d7c-b4a1-1683a417ff43 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:21.896378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254dabb-3f2c-4f2e-a4ef-411362055107 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:21.896477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6910f27-704d-4d7c-b4a1-1683a417ff43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:21.898985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f254dabb-3f2c-4f2e-a4ef-411362055107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:21.918580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254dabb-3f2c-4f2e-a4ef-411362055107 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:21.941273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c6910f27-704d-4d7c-b4a1-1683a417ff43 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:21.942572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c6910f27-704d-4d7c-b4a1-1683a417ff43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:22.028704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:22.097355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40feff-8962-4fb6-a1f7-ad535ba8a6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:22.098092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40feff-8962-4fb6-a1f7-ad535ba8a6ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:22.098373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40feff-8962-4fb6-a1f7-ad535ba8a6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:22.098650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40feff-8962-4fb6-a1f7-ad535ba8a6ae admin admin] PATH_INFO: `/v3/projects/d30e01fdb067403a8da5486814ad405b/users/4bc2a9db44144eaeb601d5dc48b9c63e/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:22.119080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40feff-8962-4fb6-a1f7-ad535ba8a6ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30e01fdb067403a8da5486814ad405b, user_id=4bc2a9db44144eaeb601d5dc48b9c63e, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:22.122993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40feff-8962-4fb6-a1f7-ad535ba8a6ae admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:22.129080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 829/1650] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:22.182817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:22.197127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 822/1651] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:21 2022] PUT /identity/v3/projects/d30e01fdb067403a8da5486814ad405b/users/4bc2a9db44144eaeb601d5dc48b9c63e/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:22.198090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6910f27-704d-4d7c-b4a1-1683a417ff43 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:22.217642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6910f27-704d-4d7c-b4a1-1683a417ff43 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:22.408841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea0a56-7cd2-4c7e-9c3e-c5c7694c62ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:22.409507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0a56-7cd2-4c7e-9c3e-c5c7694c62ae admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:22.411611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0a56-7cd2-4c7e-9c3e-c5c7694c62ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:22.411934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0a56-7cd2-4c7e-9c3e-c5c7694c62ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:22.412669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea0a56-7cd2-4c7e-9c3e-c5c7694c62ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:22.418899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea0a56-7cd2-4c7e-9c3e-c5c7694c62ae admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:22.440974 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40feff-8962-4fb6-a1f7-ad535ba8a6ae admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:22.442911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fe40feff-8962-4fb6-a1f7-ad535ba8a6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:22.446819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 830/1652] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:22.450671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea0a56-7cd2-4c7e-9c3e-c5c7694c62ae admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:22.470932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea0a56-7cd2-4c7e-9c3e-c5c7694c62ae admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:22.486133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe333b-49c1-4fca-a83e-2c6d5b463c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:22.486906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe333b-49c1-4fca-a83e-2c6d5b463c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:22.487149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe333b-49c1-4fca-a83e-2c6d5b463c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:22.487427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe333b-49c1-4fca-a83e-2c6d5b463c08 admin admin] PATH_INFO: `/v3/projects/d30e01fdb067403a8da5486814ad405b/users/4bc2a9db44144eaeb601d5dc48b9c63e/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:22.495033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe333b-49c1-4fca-a83e-2c6d5b463c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30e01fdb067403a8da5486814ad405b, user_id=4bc2a9db44144eaeb601d5dc48b9c63e, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:22.497527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe333b-49c1-4fca-a83e-2c6d5b463c08 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:22.538375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 831/1653] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:22 2022] PUT /identity/v3/projects/d30e01fdb067403a8da5486814ad405b/users/4bc2a9db44144eaeb601d5dc48b9c63e/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:22:22.573259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40feff-8962-4fb6-a1f7-ad535ba8a6ae admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:22.647138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d245851-fca4-489c-b574-2a82fc5f4b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:22.648720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d245851-fca4-489c-b574-2a82fc5f4b9b service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:22.649766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d245851-fca4-489c-b574-2a82fc5f4b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:22.650210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d245851-fca4-489c-b574-2a82fc5f4b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:22.742242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d245851-fca4-489c-b574-2a82fc5f4b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:22.744282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d245851-fca4-489c-b574-2a82fc5f4b9b service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:22.763534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 823/1654] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:24.597218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe333b-49c1-4fca-a83e-2c6d5b463c08 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:24.599232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c6fe333b-49c1-4fca-a83e-2c6d5b463c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:24.631321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe333b-49c1-4fca-a83e-2c6d5b463c08 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:24.651730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:24.652868 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:24.653244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:24.654459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] PATH_INFO: `/v3/projects/977a88ff5e02414cbb61184a82d3b850` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:24.657049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] RBAC: Authorizing `identity:delete_project(project_id=977a88ff5e02414cbb61184a82d3b850)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:24.659007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:24.851282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977a88ff5e02414cbb61184a82d3b850', 'request_id': 'req-13011ded-d711-4dc3-a547-e17aa82a905a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:24.867393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977a88ff5e02414cbb61184a82d3b850', 'request_id': 'req-13011ded-d711-4dc3-a547-e17aa82a905a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:24.959123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977a88ff5e02414cbb61184a82d3b850', 'request_id': 'req-13011ded-d711-4dc3-a547-e17aa82a905a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:24.985525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] The token cache is being invalidate because project 977a88ff5e02414cbb61184a82d3b850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:24.985820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:24.987336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-13011ded-d711-4dc3-a547-e17aa82a905a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:24.989895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 832/1655] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:24 2022] DELETE /identity/v3/projects/977a88ff5e02414cbb61184a82d3b850 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:24.996041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb19b-c9c1-408b-9e3e-f808ceb187e7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:24.996925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb19b-c9c1-408b-9e3e-f808ceb187e7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:24.997356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb19b-c9c1-408b-9e3e-f808ceb187e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.036542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b9deb19b-c9c1-408b-9e3e-f808ceb187e7 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:25.045725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b9deb19b-c9c1-408b-9e3e-f808ceb187e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:25.216537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 824/1656] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:25.224651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ddedb-b112-404a-8fef-321209ba1938 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:25.224936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ddedb-b112-404a-8fef-321209ba1938 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:25.225217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ddedb-b112-404a-8fef-321209ba1938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.282887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-593ddedb-b112-404a-8fef-321209ba1938 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:25.291081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-593ddedb-b112-404a-8fef-321209ba1938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:25.325613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 833/1657] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:25.327223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9deb19b-c9c1-408b-9e3e-f808ceb187e7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:25.337385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:25.349517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9deb19b-c9c1-408b-9e3e-f808ceb187e7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:25.357799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310304b1-858c-4309-b98e-71ebea4e5c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:25.359150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310304b1-858c-4309-b98e-71ebea4e5c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:25.359656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310304b1-858c-4309-b98e-71ebea4e5c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:25.360161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310304b1-858c-4309-b98e-71ebea4e5c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.361081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310304b1-858c-4309-b98e-71ebea4e5c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:25.361585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310304b1-858c-4309-b98e-71ebea4e5c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:25.367168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310304b1-858c-4309-b98e-71ebea4e5c06 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:25.419470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 825/1658] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:25.434617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ddedb-b112-404a-8fef-321209ba1938 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:25.453595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ddedb-b112-404a-8fef-321209ba1938 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:25.471334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a8270-b093-4bec-a1ab-60a0f80b3a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:25.472873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a8270-b093-4bec-a1ab-60a0f80b3a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:25.473162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a8270-b093-4bec-a1ab-60a0f80b3a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:25.473439 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a8270-b093-4bec-a1ab-60a0f80b3a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.474125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a8270-b093-4bec-a1ab-60a0f80b3a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:25.476118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a8270-b093-4bec-a1ab-60a0f80b3a05 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:25.627357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 834/1659] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:25 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:25.628765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310304b1-858c-4309-b98e-71ebea4e5c06 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:25.648763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310304b1-858c-4309-b98e-71ebea4e5c06 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:25.827244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9eee2-6fc8-4297-9885-84910eae6ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:25.827854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9eee2-6fc8-4297-9885-84910eae6ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:25.828150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9eee2-6fc8-4297-9885-84910eae6ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:25.828425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9eee2-6fc8-4297-9885-84910eae6ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:25.829111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9eee2-6fc8-4297-9885-84910eae6ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:25.833522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9eee2-6fc8-4297-9885-84910eae6ca8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:25.970275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 826/1660] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:25 2022] POST /identity/v3/users => generated 293 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:25.972999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a8270-b093-4bec-a1ab-60a0f80b3a05 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:25.990469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a8270-b093-4bec-a1ab-60a0f80b3a05 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:26.027037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7940c-0530-4b53-9476-9dd82069f8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:26.027037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7940c-0530-4b53-9476-9dd82069f8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:26.027037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7940c-0530-4b53-9476-9dd82069f8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:26.027037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7940c-0530-4b53-9476-9dd82069f8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:26.027685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7940c-0530-4b53-9476-9dd82069f8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:26.027685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7940c-0530-4b53-9476-9dd82069f8d5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:26.054184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:26.069175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 835/1661] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:26.073304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9eee2-6fc8-4297-9885-84910eae6ca8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:26.089682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9eee2-6fc8-4297-9885-84910eae6ca8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:26.100783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4412db-c116-4b59-adba-7894bd75f2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:26.103029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4412db-c116-4b59-adba-7894bd75f2df admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:26.103989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4412db-c116-4b59-adba-7894bd75f2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:26.104292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4412db-c116-4b59-adba-7894bd75f2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:26.105928 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4412db-c116-4b59-adba-7894bd75f2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:26.110900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4412db-c116-4b59-adba-7894bd75f2df admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:26.128858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 827/1662] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:26.129425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7940c-0530-4b53-9476-9dd82069f8d5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:26.143338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7940c-0530-4b53-9476-9dd82069f8d5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:26.165707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c7ace-5d3b-4507-887f-448db6d4a62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:26.169203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c7ace-5d3b-4507-887f-448db6d4a62b admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:26.171184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c7ace-5d3b-4507-887f-448db6d4a62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:26.171631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c7ace-5d3b-4507-887f-448db6d4a62b admin admin] PATH_INFO: `/v3/projects/a7d5e4865238425aa8b455849a746dfa/users/4ab2ba230a26415e9408f0c6b206cdfc/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:26.226077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c7ace-5d3b-4507-887f-448db6d4a62b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7d5e4865238425aa8b455849a746dfa, user_id=4ab2ba230a26415e9408f0c6b206cdfc, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:26.240963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c7ace-5d3b-4507-887f-448db6d4a62b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:26.421492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 836/1663] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:26 2022] PUT /identity/v3/projects/a7d5e4865238425aa8b455849a746dfa/users/4ab2ba230a26415e9408f0c6b206cdfc/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:26.433720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c973f7-bdbb-4c22-8b56-1e8684a1ef50 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:26.434012 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c973f7-bdbb-4c22-8b56-1e8684a1ef50 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:26.434348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c973f7-bdbb-4c22-8b56-1e8684a1ef50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:26.499040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c7c973f7-bdbb-4c22-8b56-1e8684a1ef50 None None] MFA Rules not processed for user `4ab2ba230a26415e9408f0c6b206cdfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:26.500860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c7c973f7-bdbb-4c22-8b56-1e8684a1ef50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:26.693960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 828/1664] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:26.697763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c7ace-5d3b-4507-887f-448db6d4a62b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:26.969860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c7ace-5d3b-4507-887f-448db6d4a62b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:26.978847 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590c96b-f43e-4deb-8bc4-1f149722e498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:26.979748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590c96b-f43e-4deb-8bc4-1f149722e498 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:26.980217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590c96b-f43e-4deb-8bc4-1f149722e498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:26.980636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590c96b-f43e-4deb-8bc4-1f149722e498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:26.982119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590c96b-f43e-4deb-8bc4-1f149722e498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:26.984626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590c96b-f43e-4deb-8bc4-1f149722e498 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:27.090513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c973f7-bdbb-4c22-8b56-1e8684a1ef50 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:27.092642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c7c973f7-bdbb-4c22-8b56-1e8684a1ef50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:27.118511 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 837/1665] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:26 2022] POST /identity/v3/projects => generated 387 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:27.119672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590c96b-f43e-4deb-8bc4-1f149722e498 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:27.126373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c973f7-bdbb-4c22-8b56-1e8684a1ef50 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:27.306607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:27.307593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:27.308010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:27.308384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] PATH_INFO: `/v3/users/702b55c0640047af91e739ce22d48098` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:27.310570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=702b55c0640047af91e739ce22d48098)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:27.317689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:27.326093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590c96b-f43e-4deb-8bc4-1f149722e498 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:27.326273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:27.364757 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05012432-d904-4d14-b7e6-045afe91b5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:27.365636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05012432-d904-4d14-b7e6-045afe91b5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:27.367282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05012432-d904-4d14-b7e6-045afe91b5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:27.368366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05012432-d904-4d14-b7e6-045afe91b5f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:27.369255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05012432-d904-4d14-b7e6-045afe91b5f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:27.371414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05012432-d904-4d14-b7e6-045afe91b5f2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:27.473977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 838/1666] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:27 2022] POST /identity/v3/users => generated 294 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:27.476437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d87db-8624-4ff6-ab55-4eddd0273065 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:27.477006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d87db-8624-4ff6-ab55-4eddd0273065 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:27.477495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d87db-8624-4ff6-ab55-4eddd0273065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:27.592317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6a6d87db-8624-4ff6-ab55-4eddd0273065 None None] MFA Rules not processed for user `4bc2a9db44144eaeb601d5dc48b9c63e`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:27.593862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6a6d87db-8624-4ff6-ab55-4eddd0273065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:27.621254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702b55c0640047af91e739ce22d48098'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:27.660195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702b55c0640047af91e739ce22d48098'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:27.741600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702b55c0640047af91e739ce22d48098'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:27.745598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702b55c0640047af91e739ce22d48098'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:27.770882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702b55c0640047af91e739ce22d48098'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:27.808673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 829/1667] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:27 2022] DELETE /identity/v3/users/702b55c0640047af91e739ce22d48098 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:27.808673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:27.812665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:27.835358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 839/1668] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:27.839737 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c34844-5c02-4e0a-aeb0-a3fdd3ad4a8b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:27.841267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c34844-5c02-4e0a-aeb0-a3fdd3ad4a8b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:27.841550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c34844-5c02-4e0a-aeb0-a3fdd3ad4a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:27.912635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-05c34844-5c02-4e0a-aeb0-a3fdd3ad4a8b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:27.918221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-05c34844-5c02-4e0a-aeb0-a3fdd3ad4a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:27.999547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62160ed7-d986-47de-950c-4f230dde0378 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:28.018243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54dcfc-28fd-4d7b-8c06-8137fe240a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:28.018898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54dcfc-28fd-4d7b-8c06-8137fe240a72 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.019460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54dcfc-28fd-4d7b-8c06-8137fe240a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.019744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54dcfc-28fd-4d7b-8c06-8137fe240a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.020516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54dcfc-28fd-4d7b-8c06-8137fe240a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:28.022201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54dcfc-28fd-4d7b-8c06-8137fe240a72 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:28.053388 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 830/1669] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:28.058410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54dcfc-28fd-4d7b-8c06-8137fe240a72 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:28.076226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:28.084545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54dcfc-28fd-4d7b-8c06-8137fe240a72 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:28.098796 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28819c2-df54-457f-9a15-0ff3e9138c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:28.099442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28819c2-df54-457f-9a15-0ff3e9138c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.099631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28819c2-df54-457f-9a15-0ff3e9138c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.100291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28819c2-df54-457f-9a15-0ff3e9138c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.101528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28819c2-df54-457f-9a15-0ff3e9138c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:28.106766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28819c2-df54-457f-9a15-0ff3e9138c47 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:28.111863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 840/1670] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:28.123659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 831/1671] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:28.128156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c34844-5c02-4e0a-aeb0-a3fdd3ad4a8b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:28.144002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c34844-5c02-4e0a-aeb0-a3fdd3ad4a8b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:28.155603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e3452-1dbf-49c9-afca-6c6d2883ceb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:28.157106 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e3452-1dbf-49c9-afca-6c6d2883ceb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.157405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e3452-1dbf-49c9-afca-6c6d2883ceb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.157686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e3452-1dbf-49c9-afca-6c6d2883ceb2 admin admin] PATH_INFO: `/v3/projects/f00ff8618a854d1fa241319ba6ce428e/users/eb177537f420413da5f497ff0617a6c0/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.202990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e3452-1dbf-49c9-afca-6c6d2883ceb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00ff8618a854d1fa241319ba6ce428e, user_id=eb177537f420413da5f497ff0617a6c0, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:28.230440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e3452-1dbf-49c9-afca-6c6d2883ceb2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:28.248222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28819c2-df54-457f-9a15-0ff3e9138c47 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:28.251808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a28819c2-df54-457f-9a15-0ff3e9138c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:28.364465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 841/1672] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:28 2022] PUT /identity/v3/projects/f00ff8618a854d1fa241319ba6ce428e/users/eb177537f420413da5f497ff0617a6c0/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:28.379249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba961739-e529-4793-ba6b-01caa0caed27 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.379249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba961739-e529-4793-ba6b-01caa0caed27 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.379557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba961739-e529-4793-ba6b-01caa0caed27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.438192 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28819c2-df54-457f-9a15-0ff3e9138c47 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:28.471039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ba961739-e529-4793-ba6b-01caa0caed27 None None] MFA Rules not processed for user `eb177537f420413da5f497ff0617a6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:28.472312 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ba961739-e529-4793-ba6b-01caa0caed27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:28.503489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:28.577535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e9bdd-6eb8-4728-9935-192941421d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:28.578316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e9bdd-6eb8-4728-9935-192941421d8d service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.578657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e9bdd-6eb8-4728-9935-192941421d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.578931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e9bdd-6eb8-4728-9935-192941421d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.698352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 842/1673] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:22:28.699809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba961739-e529-4793-ba6b-01caa0caed27 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:28.752296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e9bdd-6eb8-4728-9935-192941421d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:28.757477 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e9bdd-6eb8-4728-9935-192941421d8d service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:28.791992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 832/1674] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:28.853347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba961739-e529-4793-ba6b-01caa0caed27 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:28.862142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da7fe1-ed82-4875-bd91-74825d8886a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:28.862798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da7fe1-ed82-4875-bd91-74825d8886a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:28.863085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da7fe1-ed82-4875-bd91-74825d8886a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:28.865775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da7fe1-ed82-4875-bd91-74825d8886a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:28.869199 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da7fe1-ed82-4875-bd91-74825d8886a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:28.876957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da7fe1-ed82-4875-bd91-74825d8886a6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:29.050290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:29.050916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 843/1675] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:28 2022] POST /identity/v3/projects => generated 385 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:29.053107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e9bdd-6eb8-4728-9935-192941421d8d service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:29.067757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e9bdd-6eb8-4728-9935-192941421d8d service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:29.215790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e72d6c-0521-4299-9381-5adfcc534907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:29.216537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e72d6c-0521-4299-9381-5adfcc534907 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:29.216926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e72d6c-0521-4299-9381-5adfcc534907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:29.217274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e72d6c-0521-4299-9381-5adfcc534907 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:29.218037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e72d6c-0521-4299-9381-5adfcc534907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:29.221921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e72d6c-0521-4299-9381-5adfcc534907 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:29.318040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 833/1676] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:29 2022] POST /identity/v3/users => generated 299 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:29.324220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da7fe1-ed82-4875-bd91-74825d8886a6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:29.340625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da7fe1-ed82-4875-bd91-74825d8886a6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:29.360119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0ebe3-5519-42b7-abe7-934a5c26fd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:29.361003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0ebe3-5519-42b7-abe7-934a5c26fd42 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:29.361447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0ebe3-5519-42b7-abe7-934a5c26fd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:29.361876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0ebe3-5519-42b7-abe7-934a5c26fd42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:29.362891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0ebe3-5519-42b7-abe7-934a5c26fd42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:29.364704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0ebe3-5519-42b7-abe7-934a5c26fd42 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:29.401346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e72d6c-0521-4299-9381-5adfcc534907 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:29.408272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 844/1677] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:29.413936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e72d6c-0521-4299-9381-5adfcc534907 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:29.428231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265c7ef-49fe-4014-886b-69f58600bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:29.429186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265c7ef-49fe-4014-886b-69f58600bf1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:29.429615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265c7ef-49fe-4014-886b-69f58600bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:29.430057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265c7ef-49fe-4014-886b-69f58600bf1c admin admin] PATH_INFO: `/v3/projects/d2e137f7d34c45728108e69d0dadc45d/users/c5c5ec00e82a40639e9429f32559023e/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:29.470353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265c7ef-49fe-4014-886b-69f58600bf1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e137f7d34c45728108e69d0dadc45d, user_id=c5c5ec00e82a40639e9429f32559023e, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:29.474642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3265c7ef-49fe-4014-886b-69f58600bf1c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:29.600051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 834/1678] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:29 2022] PUT /identity/v3/projects/d2e137f7d34c45728108e69d0dadc45d/users/c5c5ec00e82a40639e9429f32559023e/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:29.604712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0ebe3-5519-42b7-abe7-934a5c26fd42 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:29.637158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0ebe3-5519-42b7-abe7-934a5c26fd42 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:29.757228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b56b2-8987-4da6-90f3-7082abe9dfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:29.759388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b56b2-8987-4da6-90f3-7082abe9dfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:29.759718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b56b2-8987-4da6-90f3-7082abe9dfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:29.760421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b56b2-8987-4da6-90f3-7082abe9dfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:29.761950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b56b2-8987-4da6-90f3-7082abe9dfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:29.766896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b56b2-8987-4da6-90f3-7082abe9dfb3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:29.791367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 845/1679] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:29.795121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265c7ef-49fe-4014-886b-69f58600bf1c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:29.809091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265c7ef-49fe-4014-886b-69f58600bf1c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:29.823407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c27622-7969-4486-b179-46a7318b1b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:29.824157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c27622-7969-4486-b179-46a7318b1b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:29.824447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c27622-7969-4486-b179-46a7318b1b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:29.825400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c27622-7969-4486-b179-46a7318b1b1e admin admin] PATH_INFO: `/v3/projects/d2e137f7d34c45728108e69d0dadc45d/users/c5c5ec00e82a40639e9429f32559023e/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:29.829202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c27622-7969-4486-b179-46a7318b1b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e137f7d34c45728108e69d0dadc45d, user_id=c5c5ec00e82a40639e9429f32559023e, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:29.834492 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c27622-7969-4486-b179-46a7318b1b1e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:29.857218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 835/1680] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:29 2022] PUT /identity/v3/projects/d2e137f7d34c45728108e69d0dadc45d/users/c5c5ec00e82a40639e9429f32559023e/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:29.867369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264cf08-5342-474d-acdc-d43f3eb551ce None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:29.868594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264cf08-5342-474d-acdc-d43f3eb551ce None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:29.868594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264cf08-5342-474d-acdc-d43f3eb551ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:29.913591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3264cf08-5342-474d-acdc-d43f3eb551ce None None] MFA Rules not processed for user `c5c5ec00e82a40639e9429f32559023e`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:29.915381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3264cf08-5342-474d-acdc-d43f3eb551ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:30.055191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 846/1681] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:30.068364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c27622-7969-4486-b179-46a7318b1b1e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:30.071515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-86c27622-7969-4486-b179-46a7318b1b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:30.110512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:30.199213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c27622-7969-4486-b179-46a7318b1b1e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:30.208658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc70fc3-d240-41e1-80e7-ac543e3dbebf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:30.209304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc70fc3-d240-41e1-80e7-ac543e3dbebf service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:30.209825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc70fc3-d240-41e1-80e7-ac543e3dbebf service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:30.210434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc70fc3-d240-41e1-80e7-ac543e3dbebf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:30.233382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc70fc3-d240-41e1-80e7-ac543e3dbebf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:30.236778 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc70fc3-d240-41e1-80e7-ac543e3dbebf service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:30.261473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 836/1682] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:30.460592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0e2d9-1628-408b-adf5-5b7fe22e5ec1 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:30.462318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0e2d9-1628-408b-adf5-5b7fe22e5ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:30.462746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0e2d9-1628-408b-adf5-5b7fe22e5ec1 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:30.466125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 847/1683] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:22:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:30.502302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d7569-7ae3-454f-90a8-a90f142e4b87 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:30.502831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d7569-7ae3-454f-90a8-a90f142e4b87 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:30.504011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d7569-7ae3-454f-90a8-a90f142e4b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:30.546907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-161d7569-7ae3-454f-90a8-a90f142e4b87 None None] MFA Rules not processed for user `8361f2f9c70b4a3288a187cb2d97d415`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:30.549039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-161d7569-7ae3-454f-90a8-a90f142e4b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:30.748275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 837/1684] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:22:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:30.795545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0e2d9-1628-408b-adf5-5b7fe22e5ec1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:30.797382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-99e0e2d9-1628-408b-adf5-5b7fe22e5ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:30.874188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:30.923565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0e2d9-1628-408b-adf5-5b7fe22e5ec1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:30.929000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:30.930415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:30.931153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:30.931691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] PATH_INFO: `/v3/projects/0f149f11ee014e6baf15c98444f91075` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:30.939422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f149f11ee014e6baf15c98444f91075)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:30.942617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:31.018359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d7569-7ae3-454f-90a8-a90f142e4b87 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:31.019951 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-161d7569-7ae3-454f-90a8-a90f142e4b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:31.072677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d7569-7ae3-454f-90a8-a90f142e4b87 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:31.079143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c72d34-5f0e-4246-94fd-1b7d628a1723 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:31.080593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c72d34-5f0e-4246-94fd-1b7d628a1723 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.081135 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c72d34-5f0e-4246-94fd-1b7d628a1723 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.081900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c72d34-5f0e-4246-94fd-1b7d628a1723 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.154314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c72d34-5f0e-4246-94fd-1b7d628a1723 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:31.156262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c72d34-5f0e-4246-94fd-1b7d628a1723 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:31.168921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f149f11ee014e6baf15c98444f91075', 'request_id': 'req-fb61fdb1-ace9-4f85-96e0-8513b12dac09'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:31.230554 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f149f11ee014e6baf15c98444f91075', 'request_id': 'req-fb61fdb1-ace9-4f85-96e0-8513b12dac09'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:31.236352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 838/1685] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:31.238684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c2e10-4c82-4d6f-a877-2280b8c854f4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.244638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c2e10-4c82-4d6f-a877-2280b8c854f4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.244638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c2e10-4c82-4d6f-a877-2280b8c854f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.286433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a58c2e10-4c82-4d6f-a877-2280b8c854f4 None None] MFA Rules not processed for user `1c96cc61c2b84b3b81ed6c6738f9773e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:31.288088 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a58c2e10-4c82-4d6f-a877-2280b8c854f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:31.321963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f149f11ee014e6baf15c98444f91075', 'request_id': 'req-fb61fdb1-ace9-4f85-96e0-8513b12dac09'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:31.349318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] The token cache is being invalidate because project 0f149f11ee014e6baf15c98444f91075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:31.349624 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:31.353417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:31.358626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 848/1686] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:30 2022] DELETE /identity/v3/projects/0f149f11ee014e6baf15c98444f91075 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:31.361449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:31.365662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:31.439264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 839/1687] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:31.454828 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c41b-fff0-4653-a4cb-f514ca99035a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.455366 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c41b-fff0-4653-a4cb-f514ca99035a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.455838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c41b-fff0-4653-a4cb-f514ca99035a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.490330 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-26e2c41b-fff0-4653-a4cb-f514ca99035a None None] MFA Rules not processed for user `1c96cc61c2b84b3b81ed6c6738f9773e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:31.495746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-26e2c41b-fff0-4653-a4cb-f514ca99035a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:31.508991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 840/1688] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:31.522590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c41b-fff0-4653-a4cb-f514ca99035a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:31.527337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-26e2c41b-fff0-4653-a4cb-f514ca99035a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:31.580806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61fdb1-ace9-4f85-96e0-8513b12dac09 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:31.597218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:31.598696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.599836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.600225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] PATH_INFO: `/v3/users/f4aecb224d9c46bd9a9bf2a7cfcd4078` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.604829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4aecb224d9c46bd9a9bf2a7cfcd4078)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:31.611893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:31.620601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:31.730343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c41b-fff0-4653-a4cb-f514ca99035a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:31.744383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529fec8-d094-45f4-a9d2-933c63d7a972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:31.745069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529fec8-d094-45f4-a9d2-933c63d7a972 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:31.745352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529fec8-d094-45f4-a9d2-933c63d7a972 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:31.745630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529fec8-d094-45f4-a9d2-933c63d7a972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:31.770698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e529fec8-d094-45f4-a9d2-933c63d7a972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:31.772530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e529fec8-d094-45f4-a9d2-933c63d7a972 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:31.816703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 841/1689] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:31.951711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4aecb224d9c46bd9a9bf2a7cfcd4078'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:31.979024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4aecb224d9c46bd9a9bf2a7cfcd4078'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:32.009323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1acc26-63d8-4e27-bb57-9eeb9fe362b5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:32.009607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1acc26-63d8-4e27-bb57-9eeb9fe362b5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:32.009957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1acc26-63d8-4e27-bb57-9eeb9fe362b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:32.071203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3b1acc26-63d8-4e27-bb57-9eeb9fe362b5 None None] MFA Rules not processed for user `90f54ffc10a447d0a873f3e7c6ab2647`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:32.073417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3b1acc26-63d8-4e27-bb57-9eeb9fe362b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:32.107756 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4aecb224d9c46bd9a9bf2a7cfcd4078'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:32.108435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4aecb224d9c46bd9a9bf2a7cfcd4078'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:32.109103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-235cb863-8eab-4f10-b481-15dc320006bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4aecb224d9c46bd9a9bf2a7cfcd4078'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:32.168066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:32.180830 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 849/1690] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:31 2022] DELETE /identity/v3/users/f4aecb224d9c46bd9a9bf2a7cfcd4078 => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:32.254027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 842/1691] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:32.254433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef2fe2-70a5-4f10-b212-ce0876ddd505 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:32.254433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef2fe2-70a5-4f10-b212-ce0876ddd505 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:32.254433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef2fe2-70a5-4f10-b212-ce0876ddd505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:32.307151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b2ef2fe2-70a5-4f10-b212-ce0876ddd505 None None] MFA Rules not processed for user `90f54ffc10a447d0a873f3e7c6ab2647`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:32.308433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b2ef2fe2-70a5-4f10-b212-ce0876ddd505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:32.416922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1acc26-63d8-4e27-bb57-9eeb9fe362b5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:32.418479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3b1acc26-63d8-4e27-bb57-9eeb9fe362b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:32.478584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 850/1692] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:32.491159 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef2fe2-70a5-4f10-b212-ce0876ddd505 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:32.492632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b2ef2fe2-70a5-4f10-b212-ce0876ddd505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:32.600377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1acc26-63d8-4e27-bb57-9eeb9fe362b5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:32.651414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef2fe2-70a5-4f10-b212-ce0876ddd505 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:32.657891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5cd6b-fb3b-431b-b654-3b2b80ef8958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:32.658537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5cd6b-fb3b-431b-b654-3b2b80ef8958 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:32.658827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5cd6b-fb3b-431b-b654-3b2b80ef8958 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:32.659086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5cd6b-fb3b-431b-b654-3b2b80ef8958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:32.673718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5cd6b-fb3b-431b-b654-3b2b80ef8958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:32.675468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e5cd6b-fb3b-431b-b654-3b2b80ef8958 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:32.688866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 851/1693] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:22:32.720841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b417f57-be37-43fe-a2bd-9b63becd1ad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:32.722058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417f57-be37-43fe-a2bd-9b63becd1ad3 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:32.722550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417f57-be37-43fe-a2bd-9b63becd1ad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:32.723086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b417f57-be37-43fe-a2bd-9b63becd1ad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:32.725314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2b417f57-be37-43fe-a2bd-9b63becd1ad3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:32.745400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:32.841839 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b417f57-be37-43fe-a2bd-9b63becd1ad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:32.846754 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b417f57-be37-43fe-a2bd-9b63becd1ad3 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:32.866726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 843/1694] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:22:32.897716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc9c50-d962-4fe2-91dd-c06468f83894 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:32.898006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc9c50-d962-4fe2-91dd-c06468f83894 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:32.898287 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc9c50-d962-4fe2-91dd-c06468f83894 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:32.900276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 852/1695] 10.222.0.25 () {54 vars in 912 bytes} [Sat Aug 6 01:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:32.907484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9ade5-648f-407f-b1f1-97c697a423ab None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:32.907774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9ade5-648f-407f-b1f1-97c697a423ab None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:32.908081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9ade5-648f-407f-b1f1-97c697a423ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:32.984422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc9c50-d962-4fe2-91dd-c06468f83894 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:32.986498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f7bc9c50-d962-4fe2-91dd-c06468f83894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:33.044663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-59a9ade5-648f-407f-b1f1-97c697a423ab None None] MFA Rules not processed for user `2c14fb133a0f419390fb0d00c600bdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:33.045957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-59a9ade5-648f-407f-b1f1-97c697a423ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:33.212080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc9c50-d962-4fe2-91dd-c06468f83894 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:33.224995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 844/1696] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:22:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:33.225340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:33.225723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:33.226011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:33.226290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] PATH_INFO: `/v3/users/cf36db92027f4d21be64dc085f63aac2` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:33.229551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf36db92027f4d21be64dc085f63aac2)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:33.231724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:33.260501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9ade5-648f-407f-b1f1-97c697a423ab None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:33.262190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-59a9ade5-648f-407f-b1f1-97c697a423ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:33.278066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9ade5-648f-407f-b1f1-97c697a423ab None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:33.293896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd10f75-44d8-4a45-ae7a-01f4b94e5c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:33.294805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd10f75-44d8-4a45-ae7a-01f4b94e5c90 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:33.295092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd10f75-44d8-4a45-ae7a-01f4b94e5c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:33.295371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd10f75-44d8-4a45-ae7a-01f4b94e5c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:33.304286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd10f75-44d8-4a45-ae7a-01f4b94e5c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:33.306000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd10f75-44d8-4a45-ae7a-01f4b94e5c90 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:33.316040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 845/1697] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:22:33.317638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e502-92ec-42fe-ac74-fa1513523108 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:33.318224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e502-92ec-42fe-ac74-fa1513523108 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:33.318705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e502-92ec-42fe-ac74-fa1513523108 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:33.323202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 846/1698] 10.222.0.25 () {54 vars in 912 bytes} [Sat Aug 6 01:22:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:33.329896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7fbd1-1422-43dd-97dc-728ea0af90a4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:33.330224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7fbd1-1422-43dd-97dc-728ea0af90a4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:33.330508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7fbd1-1422-43dd-97dc-728ea0af90a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:33.359694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b4f7fbd1-1422-43dd-97dc-728ea0af90a4 None None] MFA Rules not processed for user `2c14fb133a0f419390fb0d00c600bdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:33.361426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4f7fbd1-1422-43dd-97dc-728ea0af90a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:33.379911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 847/1699] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:22:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:22:33.381717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7fbd1-1422-43dd-97dc-728ea0af90a4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:33.385005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4f7fbd1-1422-43dd-97dc-728ea0af90a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:33.435650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7fbd1-1422-43dd-97dc-728ea0af90a4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:33.465809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:33.466481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:33.466814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:33.467102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] PATH_INFO: `/v3/projects/bfb77c7235f74ccfb409cbe482c564ec` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:33.468577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb77c7235f74ccfb409cbe482c564ec)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:33.469976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: WARNING py.warnings [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 01:22:33.469976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: warnings.warn(msg) Aug 06 01:22:33.469976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: Aug 06 01:22:33.472558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:33.510907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf36db92027f4d21be64dc085f63aac2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:33.522590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf36db92027f4d21be64dc085f63aac2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:33.624301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf36db92027f4d21be64dc085f63aac2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:33.631705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf36db92027f4d21be64dc085f63aac2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:33.631705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf36db92027f4d21be64dc085f63aac2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:33.726564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 853/1700] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:32 2022] DELETE /identity/v3/users/cf36db92027f4d21be64dc085f63aac2 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:22:33.730418 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:33.740614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:33.753585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb77c7235f74ccfb409cbe482c564ec', 'request_id': 'req-5f13460b-374f-4eb4-8b75-ad89dab13226'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:33.821773 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb77c7235f74ccfb409cbe482c564ec', 'request_id': 'req-5f13460b-374f-4eb4-8b75-ad89dab13226'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:33.849619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb77c7235f74ccfb409cbe482c564ec', 'request_id': 'req-5f13460b-374f-4eb4-8b75-ad89dab13226'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:33.964417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee3db3-6099-46ba-866e-e6f92a7d51f8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:33.969685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd6339-ef8d-4a32-9b1b-7d588e76cd09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:33.974122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] The token cache is being invalidate because project bfb77c7235f74ccfb409cbe482c564ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:33.974757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:33.975477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd6339-ef8d-4a32-9b1b-7d588e76cd09 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:33.976220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f13460b-374f-4eb4-8b75-ad89dab13226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:33.982851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 848/1701] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:33 2022] DELETE /identity/v3/projects/bfb77c7235f74ccfb409cbe482c564ec => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:33.985219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd6339-ef8d-4a32-9b1b-7d588e76cd09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:33.986289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd6339-ef8d-4a32-9b1b-7d588e76cd09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:33.988066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7f997-de8e-4164-a232-1fd93e83e544 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:33.988363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7f997-de8e-4164-a232-1fd93e83e544 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:33.988692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7f997-de8e-4164-a232-1fd93e83e544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:33.996066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-43cd6339-ef8d-4a32-9b1b-7d588e76cd09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:34.011310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd6339-ef8d-4a32-9b1b-7d588e76cd09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:34.017518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd6339-ef8d-4a32-9b1b-7d588e76cd09 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:34.051739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-55e7f997-de8e-4164-a232-1fd93e83e544 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:34.055357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 854/1702] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:34.055734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-55e7f997-de8e-4164-a232-1fd93e83e544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:34.082552 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:34.190278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 849/1703] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:34.200112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb00dc-029a-41e3-a585-953b636a7e53 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:34.200405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb00dc-029a-41e3-a585-953b636a7e53 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:34.200688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb00dc-029a-41e3-a585-953b636a7e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:34.266679 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d3eb00dc-029a-41e3-a585-953b636a7e53 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:34.267959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d3eb00dc-029a-41e3-a585-953b636a7e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:34.295434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 855/1704] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:22:34.300521 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7f997-de8e-4164-a232-1fd93e83e544 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:34.314479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7f997-de8e-4164-a232-1fd93e83e544 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:34.335006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c581991-4367-4244-b16b-e21958a2946c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:34.335665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c581991-4367-4244-b16b-e21958a2946c admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:34.337113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c581991-4367-4244-b16b-e21958a2946c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:34.337113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c581991-4367-4244-b16b-e21958a2946c admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:34.337564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c581991-4367-4244-b16b-e21958a2946c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:34.337877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c581991-4367-4244-b16b-e21958a2946c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:34.339524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c581991-4367-4244-b16b-e21958a2946c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:34.380184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 850/1705] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:34.385546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb00dc-029a-41e3-a585-953b636a7e53 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:34.403735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb00dc-029a-41e3-a585-953b636a7e53 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:34.444667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:34.446427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d9669-cdb4-411a-858d-3540737f19ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:34.448460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d9669-cdb4-411a-858d-3540737f19ef admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:34.448750 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d9669-cdb4-411a-858d-3540737f19ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:34.449011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d9669-cdb4-411a-858d-3540737f19ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:34.449707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d9669-cdb4-411a-858d-3540737f19ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:34.451739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d9669-cdb4-411a-858d-3540737f19ef admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:34.478919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a8f9-34fe-4a00-ae69-b5f07378ca62 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:34.482062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a8f9-34fe-4a00-ae69-b5f07378ca62 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:34.482380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a8f9-34fe-4a00-ae69-b5f07378ca62 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:34.484032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 851/1706] 10.222.0.25 () {54 vars in 961 bytes} [Sat Aug 6 01:22:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:22:34.501601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ae039-a168-45ce-bcca-656a542bd2f8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:34.501897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ae039-a168-45ce-bcca-656a542bd2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:34.503476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ae039-a168-45ce-bcca-656a542bd2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:34.553160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bc8ae039-a168-45ce-bcca-656a542bd2f8 None None] MFA Rules not processed for user `8361f2f9c70b4a3288a187cb2d97d415`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:34.557350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bc8ae039-a168-45ce-bcca-656a542bd2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:34.661395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 856/1707] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:34 2022] POST /identity/v3/projects => generated 367 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:34.662447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d9669-cdb4-411a-858d-3540737f19ef admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:34.673969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-261d9669-cdb4-411a-858d-3540737f19ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:34.688623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 852/1708] 10.222.0.25 () {58 vars in 1058 bytes} [Sat Aug 6 01:22:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:34.690224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25449-03ba-4dae-a49d-4cff0e4d8b4d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:34.690776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25449-03ba-4dae-a49d-4cff0e4d8b4d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:34.691298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec25449-03ba-4dae-a49d-4cff0e4d8b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:34.714420 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5ec25449-03ba-4dae-a49d-4cff0e4d8b4d None None] MFA Rules not processed for user `1c96cc61c2b84b3b81ed6c6738f9773e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:34.715907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5ec25449-03ba-4dae-a49d-4cff0e4d8b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:34.794149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 853/1709] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:34.795012 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25449-03ba-4dae-a49d-4cff0e4d8b4d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:34.904039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d9669-cdb4-411a-858d-3540737f19ef admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:34.918980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:34.920000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:34.920470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:34.921060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] PATH_INFO: `/v3/projects/1b517eec95d74c09a3a7e21716b364c9` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:34.923973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b517eec95d74c09a3a7e21716b364c9)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:34.929583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:34.978221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec25449-03ba-4dae-a49d-4cff0e4d8b4d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:34.989053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e28a43-c2b4-481f-a02a-db4354b992eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:34.989716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e28a43-c2b4-481f-a02a-db4354b992eb admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:34.990172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e28a43-c2b4-481f-a02a-db4354b992eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:34.990358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e28a43-c2b4-481f-a02a-db4354b992eb admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:34.991061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e28a43-c2b4-481f-a02a-db4354b992eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:34.993979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e28a43-c2b4-481f-a02a-db4354b992eb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:35.114049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 854/1710] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:34 2022] POST /identity/v3/users => generated 284 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:35.115081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e28a43-c2b4-481f-a02a-db4354b992eb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:35.198199 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b517eec95d74c09a3a7e21716b364c9', 'request_id': 'req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:35.205277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b517eec95d74c09a3a7e21716b364c9', 'request_id': 'req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:35.223747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e28a43-c2b4-481f-a02a-db4354b992eb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:35.330798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b517eec95d74c09a3a7e21716b364c9', 'request_id': 'req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:35.377598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5768976-fb76-4f78-8d5c-965efd7f473d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:35.378330 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5768976-fb76-4f78-8d5c-965efd7f473d service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:35.378523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5768976-fb76-4f78-8d5c-965efd7f473d service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:35.378802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5768976-fb76-4f78-8d5c-965efd7f473d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:35.384567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b5768976-fb76-4f78-8d5c-965efd7f473d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:35.399548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] The token cache is being invalidate because project 1b517eec95d74c09a3a7e21716b364c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:35.407063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:35.408874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9940ad89-edb2-4cf0-b888-69e2fc7c6c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:35.412880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5768976-fb76-4f78-8d5c-965efd7f473d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:35.413487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 857/1711] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:34 2022] DELETE /identity/v3/projects/1b517eec95d74c09a3a7e21716b364c9 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:35.416188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5768976-fb76-4f78-8d5c-965efd7f473d service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:35.419287 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dd84d-a3da-42f5-89bd-d79306cf1e01 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:35.419745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dd84d-a3da-42f5-89bd-d79306cf1e01 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:35.423116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dd84d-a3da-42f5-89bd-d79306cf1e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:35.428222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 855/1712] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:35.430386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5768976-fb76-4f78-8d5c-965efd7f473d service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:35.432383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b5768976-fb76-4f78-8d5c-965efd7f473d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:35.493091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b85dd84d-a3da-42f5-89bd-d79306cf1e01 None None] MFA Rules not processed for user `2d417104bf4040d4ae9ccb08097b5255`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:35.495089 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b85dd84d-a3da-42f5-89bd-d79306cf1e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:35.621732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 858/1713] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:35.622348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dd84d-a3da-42f5-89bd-d79306cf1e01 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:35.630028 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b85dd84d-a3da-42f5-89bd-d79306cf1e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:35.630154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5768976-fb76-4f78-8d5c-965efd7f473d service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:35.634500 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:35.635652 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:35.635977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:35.637077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] PATH_INFO: `/v3/users/d66a58cf4ac146f39d421e7844e1303a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:35.637940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66a58cf4ac146f39d421e7844e1303a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:35.640327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:35.671154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dd84d-a3da-42f5-89bd-d79306cf1e01 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:35.689882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81091f3c-ec1d-4aa1-9a52-b7d6d0474c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:35.693656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81091f3c-ec1d-4aa1-9a52-b7d6d0474c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:35.693656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81091f3c-ec1d-4aa1-9a52-b7d6d0474c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:35.694080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81091f3c-ec1d-4aa1-9a52-b7d6d0474c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:35.698407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81091f3c-ec1d-4aa1-9a52-b7d6d0474c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:35.710878 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81091f3c-ec1d-4aa1-9a52-b7d6d0474c19 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:35.774905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 859/1714] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:35.775414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81091f3c-ec1d-4aa1-9a52-b7d6d0474c19 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:35.783811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-81091f3c-ec1d-4aa1-9a52-b7d6d0474c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:35.820174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81091f3c-ec1d-4aa1-9a52-b7d6d0474c19 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:35.828303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:35.829725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:35.829725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:35.830081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] PATH_INFO: `/v3/users/4bc2a9db44144eaeb601d5dc48b9c63e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:35.831860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bc2a9db44144eaeb601d5dc48b9c63e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:35.834270 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:35.916386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66a58cf4ac146f39d421e7844e1303a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:35.931069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66a58cf4ac146f39d421e7844e1303a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:36.028181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66a58cf4ac146f39d421e7844e1303a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:36.029050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66a58cf4ac146f39d421e7844e1303a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:36.057037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:36.062262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66a58cf4ac146f39d421e7844e1303a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:36.077321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 856/1715] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:35 2022] DELETE /identity/v3/users/d66a58cf4ac146f39d421e7844e1303a => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:36.078331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:36.079896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:36.095023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bc2a9db44144eaeb601d5dc48b9c63e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:36.116950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bc2a9db44144eaeb601d5dc48b9c63e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:36.226867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115f580-08e7-48e3-8b35-a63b012f17f5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:36.235194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f48bbc-8bc6-4e7c-827a-11348cc30a5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:36.235897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f48bbc-8bc6-4e7c-827a-11348cc30a5e service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:36.236479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f48bbc-8bc6-4e7c-827a-11348cc30a5e service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:36.236948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f48bbc-8bc6-4e7c-827a-11348cc30a5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:36.238966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-40f48bbc-8bc6-4e7c-827a-11348cc30a5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:36.244682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bc2a9db44144eaeb601d5dc48b9c63e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:36.245399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bc2a9db44144eaeb601d5dc48b9c63e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:36.246030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bc2a9db44144eaeb601d5dc48b9c63e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:36.320357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f48bbc-8bc6-4e7c-827a-11348cc30a5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:36.323959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f48bbc-8bc6-4e7c-827a-11348cc30a5e service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:36.325241 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 860/1716] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:35 2022] DELETE /identity/v3/users/4bc2a9db44144eaeb601d5dc48b9c63e => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:36.351958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 857/1717] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:36.351958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a7ab8-c5a3-47d7-9e06-4387bcf40577 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:36.352386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a7ab8-c5a3-47d7-9e06-4387bcf40577 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:36.352644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a7ab8-c5a3-47d7-9e06-4387bcf40577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:36.365658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:36.373649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:36.386586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-272a7ab8-c5a3-47d7-9e06-4387bcf40577 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:36.388025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-272a7ab8-c5a3-47d7-9e06-4387bcf40577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:36.545370 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 858/1718] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:36.565348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a7ab8-c5a3-47d7-9e06-4387bcf40577 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:36.566962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-272a7ab8-c5a3-47d7-9e06-4387bcf40577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:36.609157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:36.615586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925da3ed-2df3-4656-b384-5eaf4698a17a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:36.633200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df69a0-f08e-4a3e-b23c-45a4502a5c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:36.634139 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df69a0-f08e-4a3e-b23c-45a4502a5c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:36.639142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df69a0-f08e-4a3e-b23c-45a4502a5c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:36.639668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df69a0-f08e-4a3e-b23c-45a4502a5c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:36.641700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df69a0-f08e-4a3e-b23c-45a4502a5c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:36.650508 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df69a0-f08e-4a3e-b23c-45a4502a5c37 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:36.689501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 861/1719] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:36.694754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df69a0-f08e-4a3e-b23c-45a4502a5c37 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:36.719135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df69a0-f08e-4a3e-b23c-45a4502a5c37 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:36.731391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5e299-2bd4-4d28-935f-41740e1d9b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:36.732101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5e299-2bd4-4d28-935f-41740e1d9b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:36.733940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5e299-2bd4-4d28-935f-41740e1d9b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:36.734249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5e299-2bd4-4d28-935f-41740e1d9b26 admin admin] PATH_INFO: `/v3/projects/5a21b6296207425f94548ffd648dd953/users/6b85eae76fb64baea9b4ea762e8b775e/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:36.763621 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a7ab8-c5a3-47d7-9e06-4387bcf40577 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:36.773336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c5d3c-8639-4ae1-9d01-477ec566bcb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:36.774533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c5d3c-8639-4ae1-9d01-477ec566bcb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:36.774967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c5d3c-8639-4ae1-9d01-477ec566bcb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:36.775891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c5d3c-8639-4ae1-9d01-477ec566bcb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:36.782164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5e299-2bd4-4d28-935f-41740e1d9b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a21b6296207425f94548ffd648dd953, user_id=6b85eae76fb64baea9b4ea762e8b775e, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:36.785186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5e299-2bd4-4d28-935f-41740e1d9b26 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:36.813619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c5d3c-8639-4ae1-9d01-477ec566bcb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:36.817651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c5d3c-8639-4ae1-9d01-477ec566bcb5 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:36.842834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 859/1720] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:22:36.897066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 862/1721] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:36 2022] PUT /identity/v3/projects/5a21b6296207425f94548ffd648dd953/users/6b85eae76fb64baea9b4ea762e8b775e/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:36.912631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51627e5-516e-4c84-9ea7-2739df315b96 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:36.913309 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51627e5-516e-4c84-9ea7-2739df315b96 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:36.913664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51627e5-516e-4c84-9ea7-2739df315b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:37.052535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d51627e5-516e-4c84-9ea7-2739df315b96 None None] MFA Rules not processed for user `6b85eae76fb64baea9b4ea762e8b775e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:37.053828 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d51627e5-516e-4c84-9ea7-2739df315b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:37.367489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 860/1722] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:37.370900 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5e299-2bd4-4d28-935f-41740e1d9b26 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:37.502253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51627e5-516e-4c84-9ea7-2739df315b96 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:37.503985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d51627e5-516e-4c84-9ea7-2739df315b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:37.570907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5e299-2bd4-4d28-935f-41740e1d9b26 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:37.626599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae815c0-f3ef-4fc4-bae6-b88b996d27f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:37.627925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae815c0-f3ef-4fc4-bae6-b88b996d27f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:37.628365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae815c0-f3ef-4fc4-bae6-b88b996d27f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:37.629361 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae815c0-f3ef-4fc4-bae6-b88b996d27f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:37.632042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae815c0-f3ef-4fc4-bae6-b88b996d27f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:37.636662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae815c0-f3ef-4fc4-bae6-b88b996d27f8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:37.745177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51627e5-516e-4c84-9ea7-2739df315b96 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:37.750167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:37.750833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:37.751120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:37.751398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] PATH_INFO: `/v3/projects/d30e01fdb067403a8da5486814ad405b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:37.754969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30e01fdb067403a8da5486814ad405b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:37.757007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:37.829084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:37.835821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 863/1723] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:37 2022] POST /identity/v3/projects => generated 369 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:37.838419 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae815c0-f3ef-4fc4-bae6-b88b996d27f8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:38.011387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30e01fdb067403a8da5486814ad405b', 'request_id': 'req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:38.025156 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30e01fdb067403a8da5486814ad405b', 'request_id': 'req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:38.035833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30e01fdb067403a8da5486814ad405b', 'request_id': 'req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:38.039948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae815c0-f3ef-4fc4-bae6-b88b996d27f8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:38.146107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] The token cache is being invalidate because project d30e01fdb067403a8da5486814ad405b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:38.146411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:38.148455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f3d7abcf-1490-4639-bb6e-ad90ebc7aedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:38.154075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 861/1724] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:37 2022] DELETE /identity/v3/projects/d30e01fdb067403a8da5486814ad405b => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:38.155498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2e6d2-f3c2-4b9f-84a8-dd9922f1f308 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:38.155844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2e6d2-f3c2-4b9f-84a8-dd9922f1f308 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:38.156088 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2e6d2-f3c2-4b9f-84a8-dd9922f1f308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:38.162180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688dc3b-00e7-451a-a264-16c173c8719e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:38.163079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688dc3b-00e7-451a-a264-16c173c8719e admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:38.164247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688dc3b-00e7-451a-a264-16c173c8719e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:38.165200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688dc3b-00e7-451a-a264-16c173c8719e admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:38.166252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688dc3b-00e7-451a-a264-16c173c8719e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:38.168451 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688dc3b-00e7-451a-a264-16c173c8719e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:38.219507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d1a2e6d2-f3c2-4b9f-84a8-dd9922f1f308 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:38.222619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d1a2e6d2-f3c2-4b9f-84a8-dd9922f1f308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:38.266937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 862/1725] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:38.271933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2e6d2-f3c2-4b9f-84a8-dd9922f1f308 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:38.271933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d1a2e6d2-f3c2-4b9f-84a8-dd9922f1f308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:38.284426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2e6d2-f3c2-4b9f-84a8-dd9922f1f308 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:38.289321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:38.290647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:38.290930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:38.291207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] PATH_INFO: `/v3/projects/0741f73bbd304e68a37194eb13a897f6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:38.299366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 864/1726] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:37 2022] POST /identity/v3/users => generated 291 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:38.300571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0741f73bbd304e68a37194eb13a897f6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:38.300923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f9ba-6187-43dd-9092-9cb5950d5031 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:38.301215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f9ba-6187-43dd-9092-9cb5950d5031 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:38.301492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f9ba-6187-43dd-9092-9cb5950d5031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:38.302801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:38.320270 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7966f9ba-6187-43dd-9092-9cb5950d5031 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:38.321498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7966f9ba-6187-43dd-9092-9cb5950d5031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:38.329707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 865/1727] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:38.330761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966f9ba-6187-43dd-9092-9cb5950d5031 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:38.332209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7966f9ba-6187-43dd-9092-9cb5950d5031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:38.347027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966f9ba-6187-43dd-9092-9cb5950d5031 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:38.353820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0335af-62ac-4096-af88-1c335ea1265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:38.354393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0335af-62ac-4096-af88-1c335ea1265c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:38.354768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0335af-62ac-4096-af88-1c335ea1265c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:38.355101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0335af-62ac-4096-af88-1c335ea1265c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:38.355838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0335af-62ac-4096-af88-1c335ea1265c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:38.357498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0335af-62ac-4096-af88-1c335ea1265c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:38.402608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 866/1728] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:38.419732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0335af-62ac-4096-af88-1c335ea1265c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:38.450297 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0335af-62ac-4096-af88-1c335ea1265c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:38.461061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0741f73bbd304e68a37194eb13a897f6', 'request_id': 'req-b917ecd9-b604-454c-954b-603f179042e8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:38.480165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0741f73bbd304e68a37194eb13a897f6', 'request_id': 'req-b917ecd9-b604-454c-954b-603f179042e8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:38.505278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0741f73bbd304e68a37194eb13a897f6', 'request_id': 'req-b917ecd9-b604-454c-954b-603f179042e8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:38.607249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a624717-db8f-42b7-a085-a6ab6f147c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:38.608220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624717-db8f-42b7-a085-a6ab6f147c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:38.608658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624717-db8f-42b7-a085-a6ab6f147c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:38.609069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a624717-db8f-42b7-a085-a6ab6f147c86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:38.609909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a624717-db8f-42b7-a085-a6ab6f147c86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:38.610425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a624717-db8f-42b7-a085-a6ab6f147c86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:38.612241 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a624717-db8f-42b7-a085-a6ab6f147c86 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:38.619428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] The token cache is being invalidate because project 0741f73bbd304e68a37194eb13a897f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:38.619737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:38.625052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:38.632445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 863/1729] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:38 2022] DELETE /identity/v3/projects/0741f73bbd304e68a37194eb13a897f6 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:38.632445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:38.634100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 867/1730] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:38.636742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:38.641409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f9442-c5c7-4a30-964e-ce0f96b3b0a3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:38.642895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f9442-c5c7-4a30-964e-ce0f96b3b0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:38.643331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f9442-c5c7-4a30-964e-ce0f96b3b0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:38.686015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917ecd9-b604-454c-954b-603f179042e8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:38.702833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2e45e-8ae9-4bc8-90a1-5c92605e337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:38.703530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2e45e-8ae9-4bc8-90a1-5c92605e337a admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:38.706614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2e45e-8ae9-4bc8-90a1-5c92605e337a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:38.706947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2e45e-8ae9-4bc8-90a1-5c92605e337a admin admin] PATH_INFO: `/v3/projects/4e75db73d54c47cbb4498d2109cbf55c/users/99d1a434720345c0a367e1979f67efe4/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:38.712432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-017f9442-c5c7-4a30-964e-ce0f96b3b0a3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:38.713660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-017f9442-c5c7-4a30-964e-ce0f96b3b0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:38.748709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 868/1731] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:38.750432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f9442-c5c7-4a30-964e-ce0f96b3b0a3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:38.752844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-017f9442-c5c7-4a30-964e-ce0f96b3b0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:38.757614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2e45e-8ae9-4bc8-90a1-5c92605e337a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e75db73d54c47cbb4498d2109cbf55c, user_id=99d1a434720345c0a367e1979f67efe4, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:38.760643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2e45e-8ae9-4bc8-90a1-5c92605e337a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:38.775920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f9442-c5c7-4a30-964e-ce0f96b3b0a3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:38.790571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fe780-326b-432b-a89a-b581441d4d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:38.791344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fe780-326b-432b-a89a-b581441d4d6d admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:38.791717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fe780-326b-432b-a89a-b581441d4d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:38.792139 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fe780-326b-432b-a89a-b581441d4d6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:38.793068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fe780-326b-432b-a89a-b581441d4d6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:38.795402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fe780-326b-432b-a89a-b581441d4d6d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:38.863946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 864/1732] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:38 2022] PUT /identity/v3/projects/4e75db73d54c47cbb4498d2109cbf55c/users/99d1a434720345c0a367e1979f67efe4/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:38.865881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ed065-ce13-4d3a-ac44-c45dcbdb21f5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:38.866462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ed065-ce13-4d3a-ac44-c45dcbdb21f5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:38.866903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ed065-ce13-4d3a-ac44-c45dcbdb21f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:38.922343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-166ed065-ce13-4d3a-ac44-c45dcbdb21f5 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:38.923598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-166ed065-ce13-4d3a-ac44-c45dcbdb21f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:38.984086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 869/1733] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:38 2022] POST /identity/v3/projects => generated 363 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:38.985405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fe780-326b-432b-a89a-b581441d4d6d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:39.128769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 865/1734] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:39.128769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ed065-ce13-4d3a-ac44-c45dcbdb21f5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:39.145200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ed065-ce13-4d3a-ac44-c45dcbdb21f5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:39.153052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4684eb-b271-44c3-afdb-b2241b7185fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:39.153192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fe780-326b-432b-a89a-b581441d4d6d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:39.153950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4684eb-b271-44c3-afdb-b2241b7185fe admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:39.154380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4684eb-b271-44c3-afdb-b2241b7185fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:39.154911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4684eb-b271-44c3-afdb-b2241b7185fe admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:39.156180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4684eb-b271-44c3-afdb-b2241b7185fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:39.158319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4684eb-b271-44c3-afdb-b2241b7185fe admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:39.176745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64931933-fecd-424e-94f5-450f07cd7a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:39.177457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64931933-fecd-424e-94f5-450f07cd7a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:39.177744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64931933-fecd-424e-94f5-450f07cd7a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:39.178023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64931933-fecd-424e-94f5-450f07cd7a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:39.178741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64931933-fecd-424e-94f5-450f07cd7a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:39.180626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64931933-fecd-424e-94f5-450f07cd7a80 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:39.232944 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 870/1735] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:39.236224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64931933-fecd-424e-94f5-450f07cd7a80 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:39.248682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64931933-fecd-424e-94f5-450f07cd7a80 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:39.255728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95167f-ee4b-4a9a-aafc-890587bc7425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:39.256607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95167f-ee4b-4a9a-aafc-890587bc7425 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:39.257008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95167f-ee4b-4a9a-aafc-890587bc7425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:39.259880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95167f-ee4b-4a9a-aafc-890587bc7425 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:39.261088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95167f-ee4b-4a9a-aafc-890587bc7425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:39.261503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95167f-ee4b-4a9a-aafc-890587bc7425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:39.263645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95167f-ee4b-4a9a-aafc-890587bc7425 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:39.311384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 871/1736] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:39.312850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95167f-ee4b-4a9a-aafc-890587bc7425 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:39.319609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 866/1737] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:39 2022] POST /identity/v3/users => generated 282 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:39.322685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4684eb-b271-44c3-afdb-b2241b7185fe admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:39.326814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ef4684eb-b271-44c3-afdb-b2241b7185fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:39.332332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95167f-ee4b-4a9a-aafc-890587bc7425 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:39.364230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03917c4-0677-4874-b077-2765a5f727dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:39.365216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03917c4-0677-4874-b077-2765a5f727dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:39.365522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03917c4-0677-4874-b077-2765a5f727dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:39.366233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03917c4-0677-4874-b077-2765a5f727dc admin admin] PATH_INFO: `/v3/projects/4e75db73d54c47cbb4498d2109cbf55c/users/99d1a434720345c0a367e1979f67efe4/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:39.370963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03917c4-0677-4874-b077-2765a5f727dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e75db73d54c47cbb4498d2109cbf55c, user_id=99d1a434720345c0a367e1979f67efe4, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:39.375044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03917c4-0677-4874-b077-2765a5f727dc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:39.395858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 872/1738] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:39 2022] PUT /identity/v3/projects/4e75db73d54c47cbb4498d2109cbf55c/users/99d1a434720345c0a367e1979f67efe4/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:39.414797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03917c4-0677-4874-b077-2765a5f727dc admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:39.478228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:39.522277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4684eb-b271-44c3-afdb-b2241b7185fe admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:39.523930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03917c4-0677-4874-b077-2765a5f727dc admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:39.530563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0c00e-4902-4330-b304-1268846e6c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:39.533612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0c00e-4902-4330-b304-1268846e6c4c admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:39.534183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0c00e-4902-4330-b304-1268846e6c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:39.534530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0c00e-4902-4330-b304-1268846e6c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:39.535317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0c00e-4902-4330-b304-1268846e6c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:39.537958 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0c00e-4902-4330-b304-1268846e6c4c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:39.641596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af086ad-6f37-4882-b077-5b98e236cfe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:39.642246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af086ad-6f37-4882-b077-5b98e236cfe8 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:39.642531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af086ad-6f37-4882-b077-5b98e236cfe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:39.642945 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af086ad-6f37-4882-b077-5b98e236cfe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:39.644866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9af086ad-6f37-4882-b077-5b98e236cfe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:39.709510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 873/1739] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:39 2022] POST /identity/v3/projects => generated 393 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:39.709510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0c00e-4902-4330-b304-1268846e6c4c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:39.749624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af086ad-6f37-4882-b077-5b98e236cfe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:39.751721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af086ad-6f37-4882-b077-5b98e236cfe8 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:39.791477 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 867/1740] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:39.793995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aeca4-43fa-4fed-b76c-fb64a4ee8880 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:39.794451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aeca4-43fa-4fed-b76c-fb64a4ee8880 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:39.794892 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aeca4-43fa-4fed-b76c-fb64a4ee8880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:39.869163 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5a9aeca4-43fa-4fed-b76c-fb64a4ee8880 None None] MFA Rules not processed for user `99d1a434720345c0a367e1979f67efe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:39.871400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5a9aeca4-43fa-4fed-b76c-fb64a4ee8880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:39.943153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0c00e-4902-4330-b304-1268846e6c4c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:39.970973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ce25e-2dd4-4c96-9e8c-b3c0f28fe884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:39.971688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ce25e-2dd4-4c96-9e8c-b3c0f28fe884 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:39.972055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ce25e-2dd4-4c96-9e8c-b3c0f28fe884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:39.972336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ce25e-2dd4-4c96-9e8c-b3c0f28fe884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:39.973046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ce25e-2dd4-4c96-9e8c-b3c0f28fe884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:39.977901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ce25e-2dd4-4c96-9e8c-b3c0f28fe884 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:40.000333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 874/1741] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:40.003619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ce25e-2dd4-4c96-9e8c-b3c0f28fe884 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:40.018668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 868/1742] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:40.019660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aeca4-43fa-4fed-b76c-fb64a4ee8880 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:40.032518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ce25e-2dd4-4c96-9e8c-b3c0f28fe884 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:40.036541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aeca4-43fa-4fed-b76c-fb64a4ee8880 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:40.046410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b34a9e-c2b3-4c59-a17a-81a29d1f54d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:40.047077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34a9e-c2b3-4c59-a17a-81a29d1f54d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:40.047361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34a9e-c2b3-4c59-a17a-81a29d1f54d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:40.047633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34a9e-c2b3-4c59-a17a-81a29d1f54d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:40.048483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b34a9e-c2b3-4c59-a17a-81a29d1f54d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:40.050343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b34a9e-c2b3-4c59-a17a-81a29d1f54d9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:40.056336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f165b7-98fd-4a1a-b234-2d328230b9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:40.057941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f165b7-98fd-4a1a-b234-2d328230b9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:40.058233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f165b7-98fd-4a1a-b234-2d328230b9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:40.058512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f165b7-98fd-4a1a-b234-2d328230b9b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:40.059192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f165b7-98fd-4a1a-b234-2d328230b9b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:40.061709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f165b7-98fd-4a1a-b234-2d328230b9b3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:40.076427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 869/1743] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:40.077387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b34a9e-c2b3-4c59-a17a-81a29d1f54d9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:40.086690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b5b34a9e-c2b3-4c59-a17a-81a29d1f54d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:40.212279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 875/1744] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:40 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:40.213256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f165b7-98fd-4a1a-b234-2d328230b9b3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:40.243381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b34a9e-c2b3-4c59-a17a-81a29d1f54d9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:40.244999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f165b7-98fd-4a1a-b234-2d328230b9b3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:40.248706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96754b-e63a-429b-a0ba-506d5a6b97af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:40.249548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96754b-e63a-429b-a0ba-506d5a6b97af service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:40.249980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96754b-e63a-429b-a0ba-506d5a6b97af service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:40.250257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96754b-e63a-429b-a0ba-506d5a6b97af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:40.262075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd0586-4023-4030-913f-c4d7dc849952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:40.266156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd0586-4023-4030-913f-c4d7dc849952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:40.267380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd0586-4023-4030-913f-c4d7dc849952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:40.268282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd0586-4023-4030-913f-c4d7dc849952 admin admin] PATH_INFO: `/v3/projects/41d2aac4ae6d4e65ae7f5431481b0707/users/97ebc958d3c441c0aabf991a3cb1ffce/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:40.284002 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96754b-e63a-429b-a0ba-506d5a6b97af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:40.291705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96754b-e63a-429b-a0ba-506d5a6b97af service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:40.329165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 870/1745] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:40.330058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96754b-e63a-429b-a0ba-506d5a6b97af service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:40.343741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96754b-e63a-429b-a0ba-506d5a6b97af service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:40.348682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9e9b3-e8ae-48d1-b822-7524da9f65c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:40.349570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9e9b3-e8ae-48d1-b822-7524da9f65c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:40.349864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9e9b3-e8ae-48d1-b822-7524da9f65c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:40.350147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9e9b3-e8ae-48d1-b822-7524da9f65c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:40.350862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9e9b3-e8ae-48d1-b822-7524da9f65c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:40.352760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9e9b3-e8ae-48d1-b822-7524da9f65c5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:40.365953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 871/1746] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:40.370746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9e9b3-e8ae-48d1-b822-7524da9f65c5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:40.389736 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:40.390454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd0586-4023-4030-913f-c4d7dc849952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41d2aac4ae6d4e65ae7f5431481b0707, user_id=97ebc958d3c441c0aabf991a3cb1ffce, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:40.396031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd0586-4023-4030-913f-c4d7dc849952 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:40.425317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9e9b3-e8ae-48d1-b822-7524da9f65c5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:40.452540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73168602-b547-4617-890e-e74aef2175af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:40.454262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73168602-b547-4617-890e-e74aef2175af admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:40.457435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73168602-b547-4617-890e-e74aef2175af admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:40.457754 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73168602-b547-4617-890e-e74aef2175af admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:40.458724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73168602-b547-4617-890e-e74aef2175af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:40.462766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73168602-b547-4617-890e-e74aef2175af admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:40.497069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 872/1747] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:40.501029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73168602-b547-4617-890e-e74aef2175af admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:40.519902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73168602-b547-4617-890e-e74aef2175af admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:40.534870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 876/1748] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:40 2022] PUT /identity/v3/projects/41d2aac4ae6d4e65ae7f5431481b0707/users/97ebc958d3c441c0aabf991a3cb1ffce/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:40.551598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c8b83-0b7f-4427-8d61-df5f5eb5cee5 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:40.560300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c8b83-0b7f-4427-8d61-df5f5eb5cee5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:40.560300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c8b83-0b7f-4427-8d61-df5f5eb5cee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:40.560944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb52b6e-f4d9-4857-a1eb-378b30eb7f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:40.561662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb52b6e-f4d9-4857-a1eb-378b30eb7f08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:40.564095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb52b6e-f4d9-4857-a1eb-378b30eb7f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:40.564370 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb52b6e-f4d9-4857-a1eb-378b30eb7f08 admin admin] PATH_INFO: `/v3/projects/e29d4aba79df4a73b9c4e97cebfbbec1/users/c1ef06481dd747a1a4a0e6af5d7406eb/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:40.606310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb52b6e-f4d9-4857-a1eb-378b30eb7f08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29d4aba79df4a73b9c4e97cebfbbec1, user_id=c1ef06481dd747a1a4a0e6af5d7406eb, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:40.609694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb52b6e-f4d9-4857-a1eb-378b30eb7f08 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:40.632276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0c3c8b83-0b7f-4427-8d61-df5f5eb5cee5 None None] MFA Rules not processed for user `97ebc958d3c441c0aabf991a3cb1ffce`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:40.632276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0c3c8b83-0b7f-4427-8d61-df5f5eb5cee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:40.683987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 873/1749] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:40 2022] PUT /identity/v3/projects/e29d4aba79df4a73b9c4e97cebfbbec1/users/c1ef06481dd747a1a4a0e6af5d7406eb/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:40.685262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb52b6e-f4d9-4857-a1eb-378b30eb7f08 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:40.792423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 877/1750] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:40.794104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881371a6-6b77-4285-afc3-4e096a87bdf8 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:40.794538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881371a6-6b77-4285-afc3-4e096a87bdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:40.795077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881371a6-6b77-4285-afc3-4e096a87bdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:40.809074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb52b6e-f4d9-4857-a1eb-378b30eb7f08 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:40.816385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73583a86-5365-47d4-a42f-bb94d64581d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:40.817162 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73583a86-5365-47d4-a42f-bb94d64581d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:40.817654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73583a86-5365-47d4-a42f-bb94d64581d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:40.818014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73583a86-5365-47d4-a42f-bb94d64581d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:40.820211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-73583a86-5365-47d4-a42f-bb94d64581d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:40.902542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73583a86-5365-47d4-a42f-bb94d64581d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:40.904201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73583a86-5365-47d4-a42f-bb94d64581d6 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:40.915727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-881371a6-6b77-4285-afc3-4e096a87bdf8 None None] MFA Rules not processed for user `c1ef06481dd747a1a4a0e6af5d7406eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:40.917000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-881371a6-6b77-4285-afc3-4e096a87bdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:40.918298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 874/1751] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:40.919216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73583a86-5365-47d4-a42f-bb94d64581d6 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:41.121296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73583a86-5365-47d4-a42f-bb94d64581d6 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:41.136313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b8f49-90d8-42ac-8e60-57e104f6a510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:41.137050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b8f49-90d8-42ac-8e60-57e104f6a510 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:41.137407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b8f49-90d8-42ac-8e60-57e104f6a510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:41.137610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b8f49-90d8-42ac-8e60-57e104f6a510 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:41.138315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b8f49-90d8-42ac-8e60-57e104f6a510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:41.143602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b8f49-90d8-42ac-8e60-57e104f6a510 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:41.144587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 878/1752] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:41.145548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881371a6-6b77-4285-afc3-4e096a87bdf8 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:41.158485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881371a6-6b77-4285-afc3-4e096a87bdf8 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:41.165124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cc73a-b6a9-4722-8112-9ee30a1706e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:41.165829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cc73a-b6a9-4722-8112-9ee30a1706e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:41.166118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cc73a-b6a9-4722-8112-9ee30a1706e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:41.166393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cc73a-b6a9-4722-8112-9ee30a1706e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:41.167241 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cc73a-b6a9-4722-8112-9ee30a1706e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:41.170665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cc73a-b6a9-4722-8112-9ee30a1706e5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:41.282123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 875/1753] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:40 2022] POST /identity/v3/projects => generated 363 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:41.287139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b8f49-90d8-42ac-8e60-57e104f6a510 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:41.314249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 879/1754] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:41 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:41.319532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879700df-2a86-49f1-826f-a73cfaf0ab7a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:41.319882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879700df-2a86-49f1-826f-a73cfaf0ab7a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:41.320173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879700df-2a86-49f1-826f-a73cfaf0ab7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:41.393830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b8f49-90d8-42ac-8e60-57e104f6a510 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:41.456952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-879700df-2a86-49f1-826f-a73cfaf0ab7a None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:41.458551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-879700df-2a86-49f1-826f-a73cfaf0ab7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:41.591220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98f74b-241b-411b-a90c-da1f958f523d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:41.592031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 880/1755] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:22:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:41.592747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879700df-2a86-49f1-826f-a73cfaf0ab7a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:41.593463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98f74b-241b-411b-a90c-da1f958f523d admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:41.593745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98f74b-241b-411b-a90c-da1f958f523d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:41.594022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98f74b-241b-411b-a90c-da1f958f523d admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:41.596118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98f74b-241b-411b-a90c-da1f958f523d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:41.599624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98f74b-241b-411b-a90c-da1f958f523d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:41.629833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879700df-2a86-49f1-826f-a73cfaf0ab7a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:41.642955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad513e95-b5ae-4e3d-bff4-b55689b24003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:41.643617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad513e95-b5ae-4e3d-bff4-b55689b24003 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:41.643938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad513e95-b5ae-4e3d-bff4-b55689b24003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:41.644376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad513e95-b5ae-4e3d-bff4-b55689b24003 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:41.645122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad513e95-b5ae-4e3d-bff4-b55689b24003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:41.647114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad513e95-b5ae-4e3d-bff4-b55689b24003 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:41.806672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 876/1756] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:41 2022] POST /identity/v3/users => generated 288 bytes in 520 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:22:41.810767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98f74b-241b-411b-a90c-da1f958f523d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:41.834775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 881/1757] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:41 2022] POST /identity/v3/users => generated 302 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:41.839519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad513e95-b5ae-4e3d-bff4-b55689b24003 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:41.859244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98f74b-241b-411b-a90c-da1f958f523d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:41.864281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad513e95-b5ae-4e3d-bff4-b55689b24003 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:41.870401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3ae47-69ca-4049-b9c5-6aec995359f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:41.871059 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ae47-69ca-4049-b9c5-6aec995359f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:41.871346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ae47-69ca-4049-b9c5-6aec995359f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:41.871681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3ae47-69ca-4049-b9c5-6aec995359f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:41.872443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3ae47-69ca-4049-b9c5-6aec995359f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:41.878147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3ae47-69ca-4049-b9c5-6aec995359f8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:41.878228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18da5c7-0a19-424e-8f12-a218724c4827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:41.878228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18da5c7-0a19-424e-8f12-a218724c4827 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:41.878228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18da5c7-0a19-424e-8f12-a218724c4827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:41.878228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18da5c7-0a19-424e-8f12-a218724c4827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:41.879932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18da5c7-0a19-424e-8f12-a218724c4827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:41.888738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18da5c7-0a19-424e-8f12-a218724c4827 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:41.896803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 882/1758] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:22:41.897715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3ae47-69ca-4049-b9c5-6aec995359f8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:41.918848 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3ae47-69ca-4049-b9c5-6aec995359f8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:41.927036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 877/1759] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:41.931172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18da5c7-0a19-424e-8f12-a218724c4827 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:41.934304 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d1933-e15b-4c45-b46d-573b59bfbf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:41.935012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d1933-e15b-4c45-b46d-573b59bfbf53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:41.935299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d1933-e15b-4c45-b46d-573b59bfbf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:41.935580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d1933-e15b-4c45-b46d-573b59bfbf53 admin admin] PATH_INFO: `/v3/projects/ac35b00fd43f4efd97f29731f9b02a19/users/73ed688ab29f4167bfd50e4152e48224/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:41.949954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18da5c7-0a19-424e-8f12-a218724c4827 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:41.956632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d1933-e15b-4c45-b46d-573b59bfbf53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac35b00fd43f4efd97f29731f9b02a19, user_id=73ed688ab29f4167bfd50e4152e48224, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:41.959657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d1933-e15b-4c45-b46d-573b59bfbf53 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:41.964501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc82a3-e386-44a8-95ba-358a57b1ce20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:41.965190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc82a3-e386-44a8-95ba-358a57b1ce20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:41.965478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc82a3-e386-44a8-95ba-358a57b1ce20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:41.965786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc82a3-e386-44a8-95ba-358a57b1ce20 admin admin] PATH_INFO: `/v3/projects/611281a4c012466a900cbecb27d31983/users/21d46d71c5274df7b02266dbbb0d2aa8/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:42.002378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc82a3-e386-44a8-95ba-358a57b1ce20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611281a4c012466a900cbecb27d31983, user_id=21d46d71c5274df7b02266dbbb0d2aa8, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:42.005511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc82a3-e386-44a8-95ba-358a57b1ce20 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:42.036320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:42.087984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 883/1760] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:41 2022] PUT /identity/v3/projects/ac35b00fd43f4efd97f29731f9b02a19/users/73ed688ab29f4167bfd50e4152e48224/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:42.092221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d1933-e15b-4c45-b46d-573b59bfbf53 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:42.141840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 878/1761] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:41 2022] PUT /identity/v3/projects/611281a4c012466a900cbecb27d31983/users/21d46d71c5274df7b02266dbbb0d2aa8/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:22:42.145155 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc82a3-e386-44a8-95ba-358a57b1ce20 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:42.242314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d1933-e15b-4c45-b46d-573b59bfbf53 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:42.269738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc82a3-e386-44a8-95ba-358a57b1ce20 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:42.277250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed413b20-cbfa-4b16-9e7f-ffe4ab75a3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:42.278723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed413b20-cbfa-4b16-9e7f-ffe4ab75a3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:42.279705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed413b20-cbfa-4b16-9e7f-ffe4ab75a3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:42.280291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed413b20-cbfa-4b16-9e7f-ffe4ab75a3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:42.281151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed413b20-cbfa-4b16-9e7f-ffe4ab75a3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:42.283205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed413b20-cbfa-4b16-9e7f-ffe4ab75a3c7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:42.300028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 879/1762] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:22:42.301900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed413b20-cbfa-4b16-9e7f-ffe4ab75a3c7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:42.304347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ed413b20-cbfa-4b16-9e7f-ffe4ab75a3c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:42.364402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d924e-e0b6-4a47-a0ca-482fe20cc663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:42.365283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d924e-e0b6-4a47-a0ca-482fe20cc663 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:42.365669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d924e-e0b6-4a47-a0ca-482fe20cc663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:42.366384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d924e-e0b6-4a47-a0ca-482fe20cc663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:42.368338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d924e-e0b6-4a47-a0ca-482fe20cc663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:42.372936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686d924e-e0b6-4a47-a0ca-482fe20cc663 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:42.403497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 884/1763] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:22:42.404175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:42.409750 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d924e-e0b6-4a47-a0ca-482fe20cc663 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:42.415310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d924e-e0b6-4a47-a0ca-482fe20cc663 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:42.421510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e28c9-d5f9-4fb0-b3db-0d4d343d8f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:42.422281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e28c9-d5f9-4fb0-b3db-0d4d343d8f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:42.422657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e28c9-d5f9-4fb0-b3db-0d4d343d8f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:42.423071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e28c9-d5f9-4fb0-b3db-0d4d343d8f43 admin admin] PATH_INFO: `/v3/projects/611281a4c012466a900cbecb27d31983/users/21d46d71c5274df7b02266dbbb0d2aa8/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:42.425227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e28c9-d5f9-4fb0-b3db-0d4d343d8f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611281a4c012466a900cbecb27d31983, user_id=21d46d71c5274df7b02266dbbb0d2aa8, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:42.429022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e28c9-d5f9-4fb0-b3db-0d4d343d8f43 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:42.454633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 885/1764] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:42 2022] PUT /identity/v3/projects/611281a4c012466a900cbecb27d31983/users/21d46d71c5274df7b02266dbbb0d2aa8/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:42.454633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e28c9-d5f9-4fb0-b3db-0d4d343d8f43 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:42.546566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed413b20-cbfa-4b16-9e7f-ffe4ab75a3c7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:42.594821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e28c9-d5f9-4fb0-b3db-0d4d343d8f43 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:42.601587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301da0d-1fe2-4ded-8609-0ae99ac3b141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:42.603581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301da0d-1fe2-4ded-8609-0ae99ac3b141 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:42.611872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301da0d-1fe2-4ded-8609-0ae99ac3b141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:42.611872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301da0d-1fe2-4ded-8609-0ae99ac3b141 admin admin] PATH_INFO: `/v3/projects/ac35b00fd43f4efd97f29731f9b02a19/users/73ed688ab29f4167bfd50e4152e48224/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:42.611872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301da0d-1fe2-4ded-8609-0ae99ac3b141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac35b00fd43f4efd97f29731f9b02a19, user_id=73ed688ab29f4167bfd50e4152e48224, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:42.611872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301da0d-1fe2-4ded-8609-0ae99ac3b141 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:42.632184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 886/1765] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:42 2022] PUT /identity/v3/projects/ac35b00fd43f4efd97f29731f9b02a19/users/73ed688ab29f4167bfd50e4152e48224/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:42.632184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef69e8-82e0-48d9-a2cb-39c4c3ead84c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:42.632184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef69e8-82e0-48d9-a2cb-39c4c3ead84c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:42.632184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef69e8-82e0-48d9-a2cb-39c4c3ead84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:42.671720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23f846-b971-4e97-ace8-ae518b29375b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:42.672379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23f846-b971-4e97-ace8-ae518b29375b service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:42.672672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23f846-b971-4e97-ace8-ae518b29375b service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:42.672946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23f846-b971-4e97-ace8-ae518b29375b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:42.707059 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f9ef69e8-82e0-48d9-a2cb-39c4c3ead84c None None] MFA Rules not processed for user `21d46d71c5274df7b02266dbbb0d2aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:42.710833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f9ef69e8-82e0-48d9-a2cb-39c4c3ead84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:42.857789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 887/1766] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:42.863468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52937c9-8926-4fc7-9438-a917408f69cb None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:42.867190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52937c9-8926-4fc7-9438-a917408f69cb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:42.869367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52937c9-8926-4fc7-9438-a917408f69cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:42.879281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23f846-b971-4e97-ace8-ae518b29375b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:42.885161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23f846-b971-4e97-ace8-ae518b29375b service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:42.919135 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 880/1767] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:22:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:22:42.924690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b067a04-02c7-4fc5-bfb3-7b33ac5ff310 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:42.924690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b067a04-02c7-4fc5-bfb3-7b33ac5ff310 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:42.924924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b067a04-02c7-4fc5-bfb3-7b33ac5ff310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:42.971341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4b067a04-02c7-4fc5-bfb3-7b33ac5ff310 None None] MFA Rules not processed for user `2c14fb133a0f419390fb0d00c600bdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:42.972575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4b067a04-02c7-4fc5-bfb3-7b33ac5ff310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:43.031930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f52937c9-8926-4fc7-9438-a917408f69cb None None] MFA Rules not processed for user `73ed688ab29f4167bfd50e4152e48224`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:43.033184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f52937c9-8926-4fc7-9438-a917408f69cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:43.129341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:43.144809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 881/1768] 10.222.0.36 () {58 vars in 1015 bytes} [Sat Aug 6 01:22:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:43.147448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b067a04-02c7-4fc5-bfb3-7b33ac5ff310 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:43.357863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 888/1769] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:42 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:43.358000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b067a04-02c7-4fc5-bfb3-7b33ac5ff310 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:43.359076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52937c9-8926-4fc7-9438-a917408f69cb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:43.363524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f52937c9-8926-4fc7-9438-a917408f69cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:43.374233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca973063-7372-43b2-8bf2-69ed38634824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:43.374922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca973063-7372-43b2-8bf2-69ed38634824 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:43.375290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca973063-7372-43b2-8bf2-69ed38634824 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:43.375574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca973063-7372-43b2-8bf2-69ed38634824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:43.404512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52937c9-8926-4fc7-9438-a917408f69cb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:43.419449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba49fa6-8353-42f1-8868-65877cf3debd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:43.422062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba49fa6-8353-42f1-8868-65877cf3debd service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:43.423670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba49fa6-8353-42f1-8868-65877cf3debd service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:43.427686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca973063-7372-43b2-8bf2-69ed38634824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:43.434317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba49fa6-8353-42f1-8868-65877cf3debd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:43.439772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca973063-7372-43b2-8bf2-69ed38634824 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:43.461759 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba49fa6-8353-42f1-8868-65877cf3debd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:43.465982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 882/1770] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:43.467054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca973063-7372-43b2-8bf2-69ed38634824 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:43.468225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba49fa6-8353-42f1-8868-65877cf3debd service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:43.484373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca973063-7372-43b2-8bf2-69ed38634824 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:43.495525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f0d82-abd4-42d2-b69e-27b65c431caf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:43.500364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 889/1771] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:43.500548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0d82-abd4-42d2-b69e-27b65c431caf service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:43.500651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0d82-abd4-42d2-b69e-27b65c431caf service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:43.500918 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0d82-abd4-42d2-b69e-27b65c431caf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:43.522627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f0d82-abd4-42d2-b69e-27b65c431caf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:43.524735 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f0d82-abd4-42d2-b69e-27b65c431caf service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:43.567702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 883/1772] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:43 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:43.813124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba49fa6-8353-42f1-8868-65877cf3debd service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:43.823588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba49fa6-8353-42f1-8868-65877cf3debd service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:43.827570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497ed32-d50b-4396-8734-4584caf3a559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:43.828315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497ed32-d50b-4396-8734-4584caf3a559 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:43.828894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497ed32-d50b-4396-8734-4584caf3a559 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:43.828894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497ed32-d50b-4396-8734-4584caf3a559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:43.942969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f0d82-abd4-42d2-b69e-27b65c431caf service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:43.955174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f0d82-abd4-42d2-b69e-27b65c431caf service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:43.971488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66369772-bd49-45c7-af29-5b69cbdb2afb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:43.972261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66369772-bd49-45c7-af29-5b69cbdb2afb service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:43.972646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66369772-bd49-45c7-af29-5b69cbdb2afb service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:43.973021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66369772-bd49-45c7-af29-5b69cbdb2afb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:43.986130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497ed32-d50b-4396-8734-4584caf3a559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:43.994598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497ed32-d50b-4396-8734-4584caf3a559 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:44.020325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 890/1773] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:44.065812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66369772-bd49-45c7-af29-5b69cbdb2afb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:44.065812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66369772-bd49-45c7-af29-5b69cbdb2afb service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:44.085731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:44.098842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 884/1774] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:44.433776 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceee002-22b6-47c5-a222-de867cde8516 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:44.434130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceee002-22b6-47c5-a222-de867cde8516 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:44.434541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceee002-22b6-47c5-a222-de867cde8516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:44.506613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bceee002-22b6-47c5-a222-de867cde8516 None None] MFA Rules not processed for user `2d417104bf4040d4ae9ccb08097b5255`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:44.506613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bceee002-22b6-47c5-a222-de867cde8516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:44.907650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 891/1775] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:45.892181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:45.950916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2be3b-dde9-4f9a-9678-24944efcde4d None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:45.951205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2be3b-dde9-4f9a-9678-24944efcde4d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:45.956532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2be3b-dde9-4f9a-9678-24944efcde4d None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:45.959122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 885/1776] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:22:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:22:46.130935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81bd7-0d2f-4967-8bc6-fade7b49a69c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:46.132049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81bd7-0d2f-4967-8bc6-fade7b49a69c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:46.132547 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81bd7-0d2f-4967-8bc6-fade7b49a69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:46.196764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e0c81bd7-0d2f-4967-8bc6-fade7b49a69c None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:46.201432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e0c81bd7-0d2f-4967-8bc6-fade7b49a69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:46.307135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:46.464844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 892/1777] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:22:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:47.241856 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec3a8f-8a56-4744-8966-966b038603f8 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:47.242387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec3a8f-8a56-4744-8966-966b038603f8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:47.242387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec3a8f-8a56-4744-8966-966b038603f8 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:47.243812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 886/1778] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:22:47.270677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c81bd7-0d2f-4967-8bc6-fade7b49a69c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:47.270677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e0c81bd7-0d2f-4967-8bc6-fade7b49a69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:47.297419 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c81bd7-0d2f-4967-8bc6-fade7b49a69c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:47.324320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1460da-2982-4c39-baf8-2107913e2e8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:47.325274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1460da-2982-4c39-baf8-2107913e2e8a service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:47.325569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1460da-2982-4c39-baf8-2107913e2e8a service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:47.325853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1460da-2982-4c39-baf8-2107913e2e8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:47.339306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76688e9a-1b82-463b-a378-f52ac18d1f84 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:47.339739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76688e9a-1b82-463b-a378-f52ac18d1f84 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:47.345666 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76688e9a-1b82-463b-a378-f52ac18d1f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:47.360702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1460da-2982-4c39-baf8-2107913e2e8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:47.363454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1460da-2982-4c39-baf8-2107913e2e8a service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:47.392483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 893/1779] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:47.407277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-76688e9a-1b82-463b-a378-f52ac18d1f84 None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:47.408716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-76688e9a-1b82-463b-a378-f52ac18d1f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:47.433803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 887/1780] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:22:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:47.598224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1460da-2982-4c39-baf8-2107913e2e8a service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:47.632736 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1460da-2982-4c39-baf8-2107913e2e8a service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:47.650235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f243116-efba-4cb2-beb6-dc62b449363a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:47.654871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f243116-efba-4cb2-beb6-dc62b449363a service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:47.655452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f243116-efba-4cb2-beb6-dc62b449363a service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:47.660327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f243116-efba-4cb2-beb6-dc62b449363a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:47.690890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f243116-efba-4cb2-beb6-dc62b449363a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:47.702108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f243116-efba-4cb2-beb6-dc62b449363a service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:47.777293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 894/1781] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:48.048560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:48.122683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76688e9a-1b82-463b-a378-f52ac18d1f84 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:48.124752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-76688e9a-1b82-463b-a378-f52ac18d1f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:48.332137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:48.332559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76688e9a-1b82-463b-a378-f52ac18d1f84 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:48.350780 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:48.354973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:48.354973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:48.354973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] PATH_INFO: `/v3/users/4ab2ba230a26415e9408f0c6b206cdfc` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:48.364875 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab2ba230a26415e9408f0c6b206cdfc)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:48.366951 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:48.610564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab2ba230a26415e9408f0c6b206cdfc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:48.624650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab2ba230a26415e9408f0c6b206cdfc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:48.721692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab2ba230a26415e9408f0c6b206cdfc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:48.722401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab2ba230a26415e9408f0c6b206cdfc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:48.746199 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab2ba230a26415e9408f0c6b206cdfc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:48.756806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 888/1782] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:48 2022] DELETE /identity/v3/users/4ab2ba230a26415e9408f0c6b206cdfc => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:48.761609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a297cd-2eec-41a8-9698-658485e2fca0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:48.761966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a297cd-2eec-41a8-9698-658485e2fca0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:48.762317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a297cd-2eec-41a8-9698-658485e2fca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:48.818023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-13a297cd-2eec-41a8-9698-658485e2fca0 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:48.819575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-13a297cd-2eec-41a8-9698-658485e2fca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:49.014621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 895/1783] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:49.044068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:49.045844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:49.209562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be461565-a75b-468d-b0e9-56c53c709dbe admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:49.219856 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d8ea5-045c-4bb4-9283-60b84e2644af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:49.220758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d8ea5-045c-4bb4-9283-60b84e2644af service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:49.222182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d8ea5-045c-4bb4-9283-60b84e2644af service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:49.223205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d8ea5-045c-4bb4-9283-60b84e2644af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:49.254583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d8ea5-045c-4bb4-9283-60b84e2644af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:49.260963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d8ea5-045c-4bb4-9283-60b84e2644af service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:49.282533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 889/1784] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:49.800263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:50.316848 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:50.681947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a297cd-2eec-41a8-9698-658485e2fca0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:50.683654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-13a297cd-2eec-41a8-9698-658485e2fca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:50.709838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a297cd-2eec-41a8-9698-658485e2fca0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:50.720322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:50.721208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:50.722526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:50.722967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] PATH_INFO: `/v3/projects/a7d5e4865238425aa8b455849a746dfa` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:50.725387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7d5e4865238425aa8b455849a746dfa)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:50.732447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:50.931893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7d5e4865238425aa8b455849a746dfa', 'request_id': 'req-17311ac3-0b49-4bc9-958a-92a4389e84f2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:50.938690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7d5e4865238425aa8b455849a746dfa', 'request_id': 'req-17311ac3-0b49-4bc9-958a-92a4389e84f2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.040265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7d5e4865238425aa8b455849a746dfa', 'request_id': 'req-17311ac3-0b49-4bc9-958a-92a4389e84f2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.057116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] The token cache is being invalidate because project a7d5e4865238425aa8b455849a746dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:51.057618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.058497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.063857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 896/1785] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:50 2022] DELETE /identity/v3/projects/a7d5e4865238425aa8b455849a746dfa => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:51.064900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d8ea5-045c-4bb4-9283-60b84e2644af service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:51.080168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d8ea5-045c-4bb4-9283-60b84e2644af service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:51.111844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:51.113794 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:51.178263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:51.179194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:51.179628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:51.180237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] PATH_INFO: `/v3/users/eb177537f420413da5f497ff0617a6c0` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:51.182888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb177537f420413da5f497ff0617a6c0)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:51.186603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:51.236550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17311ac3-0b49-4bc9-958a-92a4389e84f2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:51.245594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:51.245594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:51.245594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:51.245594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] PATH_INFO: `/v3/users/97ebc958d3c441c0aabf991a3cb1ffce` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:51.245916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ebc958d3c441c0aabf991a3cb1ffce)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:51.246165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:51.431936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb177537f420413da5f497ff0617a6c0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.439338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb177537f420413da5f497ff0617a6c0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.545378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb177537f420413da5f497ff0617a6c0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.546770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb177537f420413da5f497ff0617a6c0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.560128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ebc958d3c441c0aabf991a3cb1ffce'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.561251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-92ee4357-10d3-4e6c-ba23-86e5c603d4b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb177537f420413da5f497ff0617a6c0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.568005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ebc958d3c441c0aabf991a3cb1ffce'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.568188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 890/1786] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:51 2022] DELETE /identity/v3/users/eb177537f420413da5f497ff0617a6c0 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:22:51.670506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ebc958d3c441c0aabf991a3cb1ffce'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.671145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ebc958d3c441c0aabf991a3cb1ffce'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.671872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ebc958d3c441c0aabf991a3cb1ffce'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:51.711879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 897/1787] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:51 2022] DELETE /identity/v3/users/97ebc958d3c441c0aabf991a3cb1ffce => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:22:51.717444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799f2b7-84e7-4f67-bdd2-0d34050508ba None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:51.717941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799f2b7-84e7-4f67-bdd2-0d34050508ba None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:51.718627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799f2b7-84e7-4f67-bdd2-0d34050508ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:51.754651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4799f2b7-84e7-4f67-bdd2-0d34050508ba None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:51.759958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4799f2b7-84e7-4f67-bdd2-0d34050508ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:51.799878 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:51.801683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:51.845272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:51.910805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8a23e-3ae5-43a3-8ebb-0e8bf140e309 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:51.920336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:51.921585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:51.922179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:51.922657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] PATH_INFO: `/v3/users/6b85eae76fb64baea9b4ea762e8b775e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:51.924269 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b85eae76fb64baea9b4ea762e8b775e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:51.939259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:51.969972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 891/1788] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:51.980966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799f2b7-84e7-4f67-bdd2-0d34050508ba None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:51.983922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4799f2b7-84e7-4f67-bdd2-0d34050508ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:52.082954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799f2b7-84e7-4f67-bdd2-0d34050508ba None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:52.088281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0c167-e26d-4454-a9c8-0d67cb87edab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:52.089180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0c167-e26d-4454-a9c8-0d67cb87edab service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:52.089787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0c167-e26d-4454-a9c8-0d67cb87edab service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:52.090210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0c167-e26d-4454-a9c8-0d67cb87edab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:52.104336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0c167-e26d-4454-a9c8-0d67cb87edab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:52.106254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0c167-e26d-4454-a9c8-0d67cb87edab service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:52.114005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:52.121053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 892/1789] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:22:52.203002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b85eae76fb64baea9b4ea762e8b775e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:52.215659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b85eae76fb64baea9b4ea762e8b775e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:52.283585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0c167-e26d-4454-a9c8-0d67cb87edab service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:52.286194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1aa0c167-e26d-4454-a9c8-0d67cb87edab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:52.309980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b85eae76fb64baea9b4ea762e8b775e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:52.309980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b85eae76fb64baea9b4ea762e8b775e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:52.313559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0c167-e26d-4454-a9c8-0d67cb87edab service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:52.319762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:52.320297 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-3a86c7e2-a08b-4d92-a625-b79b9b7987be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b85eae76fb64baea9b4ea762e8b775e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:52.321217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:52.321652 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:52.321993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] PATH_INFO: `/v3/projects/f00ff8618a854d1fa241319ba6ce428e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:52.323505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00ff8618a854d1fa241319ba6ce428e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:52.326159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:52.363091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 898/1790] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:51 2022] DELETE /identity/v3/users/6b85eae76fb64baea9b4ea762e8b775e => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:22:52.370322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc650e34-e1ee-42c9-b5aa-56dbc3d44e80 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:52.376204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc650e34-e1ee-42c9-b5aa-56dbc3d44e80 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:52.376644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc650e34-e1ee-42c9-b5aa-56dbc3d44e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:52.406360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bc650e34-e1ee-42c9-b5aa-56dbc3d44e80 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:52.407680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bc650e34-e1ee-42c9-b5aa-56dbc3d44e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:52.558121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00ff8618a854d1fa241319ba6ce428e', 'request_id': 'req-eae17ba5-467d-4ca9-8389-877c88bc8ba1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:52.560625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 899/1791] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:52.562098 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc650e34-e1ee-42c9-b5aa-56dbc3d44e80 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:52.576120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bc650e34-e1ee-42c9-b5aa-56dbc3d44e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:52.596057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00ff8618a854d1fa241319ba6ce428e', 'request_id': 'req-eae17ba5-467d-4ca9-8389-877c88bc8ba1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:52.609759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00ff8618a854d1fa241319ba6ce428e', 'request_id': 'req-eae17ba5-467d-4ca9-8389-877c88bc8ba1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:52.614148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc650e34-e1ee-42c9-b5aa-56dbc3d44e80 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:52.722967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:52.724293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:52.724793 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:52.725298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] PATH_INFO: `/v3/users/c1ef06481dd747a1a4a0e6af5d7406eb` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:52.727993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ef06481dd747a1a4a0e6af5d7406eb)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:52.731263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:52.736817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] The token cache is being invalidate because project f00ff8618a854d1fa241319ba6ce428e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:52.737146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:52.749561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:52.749561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 893/1792] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:52 2022] DELETE /identity/v3/projects/f00ff8618a854d1fa241319ba6ce428e => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:52.749561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:52.749561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:52.862125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae17ba5-467d-4ca9-8389-877c88bc8ba1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:52.881045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116acc8c-6676-4a25-8e53-4bb38d23a862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:52.881686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116acc8c-6676-4a25-8e53-4bb38d23a862 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:52.882048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116acc8c-6676-4a25-8e53-4bb38d23a862 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:52.882251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116acc8c-6676-4a25-8e53-4bb38d23a862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:52.887707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-116acc8c-6676-4a25-8e53-4bb38d23a862 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:52.903429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116acc8c-6676-4a25-8e53-4bb38d23a862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:52.905785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116acc8c-6676-4a25-8e53-4bb38d23a862 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:52.918227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 894/1793] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:52.919108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116acc8c-6676-4a25-8e53-4bb38d23a862 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:52.920779 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-116acc8c-6676-4a25-8e53-4bb38d23a862 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:52.934350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116acc8c-6676-4a25-8e53-4bb38d23a862 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:52.939857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:52.941035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:52.941227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:52.941462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] PATH_INFO: `/v3/users/c5c5ec00e82a40639e9429f32559023e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:52.942914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c5ec00e82a40639e9429f32559023e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:52.944966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:53.060755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ef06481dd747a1a4a0e6af5d7406eb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.071050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ef06481dd747a1a4a0e6af5d7406eb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.157485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c5ec00e82a40639e9429f32559023e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.169889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c5ec00e82a40639e9429f32559023e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.182518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ef06481dd747a1a4a0e6af5d7406eb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.183245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:53.183677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ef06481dd747a1a4a0e6af5d7406eb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.184513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ef06481dd747a1a4a0e6af5d7406eb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.234688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 900/1794] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:52 2022] DELETE /identity/v3/users/c1ef06481dd747a1a4a0e6af5d7406eb => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:53.236293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:53.238071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:53.287001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c5ec00e82a40639e9429f32559023e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.288341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c5ec00e82a40639e9429f32559023e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.318321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6f24322c-6e2e-4a71-998f-505e8e301f8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c5ec00e82a40639e9429f32559023e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.340991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 895/1795] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:52 2022] DELETE /identity/v3/users/c5c5ec00e82a40639e9429f32559023e => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:53.344205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829608b3-73eb-49ec-853d-70991ea9978a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:53.344898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829608b3-73eb-49ec-853d-70991ea9978a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:53.345353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829608b3-73eb-49ec-853d-70991ea9978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:53.390619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-829608b3-73eb-49ec-853d-70991ea9978a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:53.393274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-829608b3-73eb-49ec-853d-70991ea9978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:53.455088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012e606-2431-4fd7-93c0-f6fd79afdec7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:53.518398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:53.628336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:53.639134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 896/1796] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:53.677869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:53.680275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829608b3-73eb-49ec-853d-70991ea9978a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:53.682088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:53.682597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:53.683071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] PATH_INFO: `/v3/projects/41d2aac4ae6d4e65ae7f5431481b0707` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:53.685349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41d2aac4ae6d4e65ae7f5431481b0707)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:53.686952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-829608b3-73eb-49ec-853d-70991ea9978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:53.688960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:53.831867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829608b3-73eb-49ec-853d-70991ea9978a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:53.856009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41d2aac4ae6d4e65ae7f5431481b0707', 'request_id': 'req-017111c3-e9d9-45e9-a654-7ae24ab34517'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.865809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41d2aac4ae6d4e65ae7f5431481b0707', 'request_id': 'req-017111c3-e9d9-45e9-a654-7ae24ab34517'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:53.947757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd77ae-1642-40c9-945d-f23ec6cf110d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:53.948620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd77ae-1642-40c9-945d-f23ec6cf110d service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:53.949032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd77ae-1642-40c9-945d-f23ec6cf110d service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:53.949355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd77ae-1642-40c9-945d-f23ec6cf110d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:54.022683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41d2aac4ae6d4e65ae7f5431481b0707', 'request_id': 'req-017111c3-e9d9-45e9-a654-7ae24ab34517'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.069144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] The token cache is being invalidate because project 41d2aac4ae6d4e65ae7f5431481b0707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:54.069633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.071462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.075646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd77ae-1642-40c9-945d-f23ec6cf110d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:54.076525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 901/1797] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:53 2022] DELETE /identity/v3/projects/41d2aac4ae6d4e65ae7f5431481b0707 => generated 0 bytes in 841 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:54.080192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:54.080329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd77ae-1642-40c9-945d-f23ec6cf110d service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:54.082560 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:54.093279 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 897/1798] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:54.106080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017111c3-e9d9-45e9-a654-7ae24ab34517 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:54.112756 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:54.118478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:54.120649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:54.121039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:54.121433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] PATH_INFO: `/v3/users/21d46d71c5274df7b02266dbbb0d2aa8` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:54.124926 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21d46d71c5274df7b02266dbbb0d2aa8)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:54.127046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:54.173483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd77ae-1642-40c9-945d-f23ec6cf110d service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:54.175944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-76fd77ae-1642-40c9-945d-f23ec6cf110d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:54.190407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd77ae-1642-40c9-945d-f23ec6cf110d service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:54.198036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:54.198935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:54.199470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:54.199962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] PATH_INFO: `/v3/projects/5a21b6296207425f94548ffd648dd953` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:54.201382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a21b6296207425f94548ffd648dd953)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:54.204355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:54.388346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21d46d71c5274df7b02266dbbb0d2aa8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.402993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21d46d71c5274df7b02266dbbb0d2aa8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.409184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a21b6296207425f94548ffd648dd953', 'request_id': 'req-3581b8eb-189c-44b1-b961-b3d0c604706c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.433758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a21b6296207425f94548ffd648dd953', 'request_id': 'req-3581b8eb-189c-44b1-b961-b3d0c604706c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.442068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a21b6296207425f94548ffd648dd953', 'request_id': 'req-3581b8eb-189c-44b1-b961-b3d0c604706c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.479989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21d46d71c5274df7b02266dbbb0d2aa8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.482309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21d46d71c5274df7b02266dbbb0d2aa8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.483218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21d46d71c5274df7b02266dbbb0d2aa8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.517639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 902/1799] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:54 2022] DELETE /identity/v3/users/21d46d71c5274df7b02266dbbb0d2aa8 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:22:54.565561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] The token cache is being invalidate because project 5a21b6296207425f94548ffd648dd953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:54.566086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.566803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.569706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 898/1800] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:54 2022] DELETE /identity/v3/projects/5a21b6296207425f94548ffd648dd953 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:54.573591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:54.575044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:54.673175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c37f70-71a4-41db-a5c6-a058c4f99214 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:54.697906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:54.698771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:54.699063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:54.702240 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] PATH_INFO: `/v3/users/99d1a434720345c0a367e1979f67efe4` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:54.714264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d1a434720345c0a367e1979f67efe4)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:54.720365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:54.814858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:54.827702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:54.936932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581b8eb-189c-44b1-b961-b3d0c604706c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:54.943827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:54.944522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:54.944914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:54.945193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] PATH_INFO: `/v3/projects/d2e137f7d34c45728108e69d0dadc45d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:54.947261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2e137f7d34c45728108e69d0dadc45d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:54.949459 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:54.968828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d1a434720345c0a367e1979f67efe4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:54.980792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d1a434720345c0a367e1979f67efe4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.074363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d1a434720345c0a367e1979f67efe4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.078576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d1a434720345c0a367e1979f67efe4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.078576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d1a434720345c0a367e1979f67efe4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.112466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 903/1801] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:54 2022] DELETE /identity/v3/users/99d1a434720345c0a367e1979f67efe4 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:55.116459 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2e137f7d34c45728108e69d0dadc45d', 'request_id': 'req-89d3733d-5f59-4a34-8fc9-c0d2cff48607'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.130293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2e137f7d34c45728108e69d0dadc45d', 'request_id': 'req-89d3733d-5f59-4a34-8fc9-c0d2cff48607'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.144096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2e137f7d34c45728108e69d0dadc45d', 'request_id': 'req-89d3733d-5f59-4a34-8fc9-c0d2cff48607'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.164121 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:55.168037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:55.239884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] The token cache is being invalidate because project d2e137f7d34c45728108e69d0dadc45d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:55.240930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.241960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.244966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 899/1802] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:54 2022] DELETE /identity/v3/projects/d2e137f7d34c45728108e69d0dadc45d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:22:55.246047 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:55.249852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:55.262820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695327e-6624-4dab-b65a-fff29e0cf4b2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:55.269058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:55.269960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:55.270407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:55.270680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] PATH_INFO: `/v3/projects/e29d4aba79df4a73b9c4e97cebfbbec1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:55.272107 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e29d4aba79df4a73b9c4e97cebfbbec1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:55.274864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:55.381813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3733d-5f59-4a34-8fc9-c0d2cff48607 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:55.404893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:55.405560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:55.405835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:55.406109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] PATH_INFO: `/v3/projects/611281a4c012466a900cbecb27d31983` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:55.407303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=611281a4c012466a900cbecb27d31983)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:55.409260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:55.412803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e29d4aba79df4a73b9c4e97cebfbbec1', 'request_id': 'req-51c4c775-56f0-4558-b475-a1db47393135'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.419590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e29d4aba79df4a73b9c4e97cebfbbec1', 'request_id': 'req-51c4c775-56f0-4558-b475-a1db47393135'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.562357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e29d4aba79df4a73b9c4e97cebfbbec1', 'request_id': 'req-51c4c775-56f0-4558-b475-a1db47393135'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.578461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] The token cache is being invalidate because project e29d4aba79df4a73b9c4e97cebfbbec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:55.579234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.580833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-51c4c775-56f0-4558-b475-a1db47393135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.585156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 904/1803] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:55 2022] DELETE /identity/v3/projects/e29d4aba79df4a73b9c4e97cebfbbec1 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:22:55.587521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940719b-a619-4949-b148-923bae41db20 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:55.588196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940719b-a619-4949-b148-923bae41db20 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:55.588745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940719b-a619-4949-b148-923bae41db20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:55.591014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:55.597811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '611281a4c012466a900cbecb27d31983', 'request_id': 'req-3624887d-3f73-4373-b97c-a1d7c8cacac6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.608752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '611281a4c012466a900cbecb27d31983', 'request_id': 'req-3624887d-3f73-4373-b97c-a1d7c8cacac6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.614077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2940719b-a619-4949-b148-923bae41db20 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:55.615454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2940719b-a619-4949-b148-923bae41db20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:55.615895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '611281a4c012466a900cbecb27d31983', 'request_id': 'req-3624887d-3f73-4373-b97c-a1d7c8cacac6'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.687612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] The token cache is being invalidate because project 611281a4c012466a900cbecb27d31983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:55.688015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.697675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 905/1804] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:55.697866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3624887d-3f73-4373-b97c-a1d7c8cacac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.697947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940719b-a619-4949-b148-923bae41db20 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:55.697947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2940719b-a619-4949-b148-923bae41db20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:55.699218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 900/1805] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:55 2022] DELETE /identity/v3/projects/611281a4c012466a900cbecb27d31983 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:55.701388 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440c480-c856-4566-bb6c-d27ff78c83da None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:55.701956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440c480-c856-4566-bb6c-d27ff78c83da None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:55.702556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440c480-c856-4566-bb6c-d27ff78c83da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:55.705527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940719b-a619-4949-b148-923bae41db20 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:55.710702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:55.711942 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:55.712226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:55.712500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] PATH_INFO: `/v3/users/73ed688ab29f4167bfd50e4152e48224` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:55.713729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ed688ab29f4167bfd50e4152e48224)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:55.715887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:55.724742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e440c480-c856-4566-bb6c-d27ff78c83da None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:55.726288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e440c480-c856-4566-bb6c-d27ff78c83da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:55.739348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 901/1806] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:55.741185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572caab1-0669-401b-8bf0-f8a3d91cae81 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:55.741185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572caab1-0669-401b-8bf0-f8a3d91cae81 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:55.747829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572caab1-0669-401b-8bf0-f8a3d91cae81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:55.762093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-572caab1-0669-401b-8bf0-f8a3d91cae81 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:55.763317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-572caab1-0669-401b-8bf0-f8a3d91cae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:55.774620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 902/1807] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:55.774620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572caab1-0669-401b-8bf0-f8a3d91cae81 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:55.784193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572caab1-0669-401b-8bf0-f8a3d91cae81 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:55.789454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2f9e6-6307-49f3-9247-b085a3d73a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:55.790323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2f9e6-6307-49f3-9247-b085a3d73a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:55.790759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2f9e6-6307-49f3-9247-b085a3d73a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:55.791083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2f9e6-6307-49f3-9247-b085a3d73a09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:55.791863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2f9e6-6307-49f3-9247-b085a3d73a09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:55.792184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2f9e6-6307-49f3-9247-b085a3d73a09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:55.793817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2f9e6-6307-49f3-9247-b085a3d73a09 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:55.829980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 903/1808] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:55.831182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcddc1-7cee-4c34-9d5b-b5a018c5a0bf None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:55.831523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcddc1-7cee-4c34-9d5b-b5a018c5a0bf None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:55.831828 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcddc1-7cee-4c34-9d5b-b5a018c5a0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:55.858128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-64fcddc1-7cee-4c34-9d5b-b5a018c5a0bf None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:55.859426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-64fcddc1-7cee-4c34-9d5b-b5a018c5a0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:55.870473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 904/1809] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:55.870473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcddc1-7cee-4c34-9d5b-b5a018c5a0bf None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:55.872040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-64fcddc1-7cee-4c34-9d5b-b5a018c5a0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:55.886082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcddc1-7cee-4c34-9d5b-b5a018c5a0bf None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:55.890901 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:55.891765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:55.892177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:55.892551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] PATH_INFO: `/v3/projects/4e75db73d54c47cbb4498d2109cbf55c` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:55.893814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e75db73d54c47cbb4498d2109cbf55c)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:55.895809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:55.926138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ed688ab29f4167bfd50e4152e48224'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:55.934080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ed688ab29f4167bfd50e4152e48224'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.048556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ed688ab29f4167bfd50e4152e48224'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.050873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:56.052681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ed688ab29f4167bfd50e4152e48224'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.053356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ed688ab29f4167bfd50e4152e48224'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.053643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e75db73d54c47cbb4498d2109cbf55c', 'request_id': 'req-e921c6ef-7a67-4413-b48a-efe80a6523cd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.061311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e75db73d54c47cbb4498d2109cbf55c', 'request_id': 'req-e921c6ef-7a67-4413-b48a-efe80a6523cd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.068073 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e75db73d54c47cbb4498d2109cbf55c', 'request_id': 'req-e921c6ef-7a67-4413-b48a-efe80a6523cd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.089463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 906/1810] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:22:55 2022] DELETE /identity/v3/users/73ed688ab29f4167bfd50e4152e48224 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:56.089463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.091994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:56.171361 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e413c-5ea2-4e4c-ad1d-6f8e7e4cb24d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:56.177290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13c4da-312e-46e1-a734-7b8fd4e566e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:56.177802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] The token cache is being invalidate because project 4e75db73d54c47cbb4498d2109cbf55c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:56.178665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13c4da-312e-46e1-a734-7b8fd4e566e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.178896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.179237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13c4da-312e-46e1-a734-7b8fd4e566e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.179572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.179915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13c4da-312e-46e1-a734-7b8fd4e566e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.180719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13c4da-312e-46e1-a734-7b8fd4e566e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:56.182787 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13c4da-312e-46e1-a734-7b8fd4e566e5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:56.186000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 905/1811] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:55 2022] DELETE /identity/v3/projects/4e75db73d54c47cbb4498d2109cbf55c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:56.187188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.189870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:56.262681 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921c6ef-7a67-4413-b48a-efe80a6523cd admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:56.267827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f4859-b5e2-4979-95dc-11a75e5e39c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:56.268450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f4859-b5e2-4979-95dc-11a75e5e39c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.268736 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f4859-b5e2-4979-95dc-11a75e5e39c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.269008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f4859-b5e2-4979-95dc-11a75e5e39c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.269702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f4859-b5e2-4979-95dc-11a75e5e39c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:56.270002 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f4859-b5e2-4979-95dc-11a75e5e39c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:56.271579 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f4859-b5e2-4979-95dc-11a75e5e39c0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:56.292015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 907/1812] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:56 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:56.293133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0446b92-8ad9-4eae-a778-483911a1c1ce None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.293525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0446b92-8ad9-4eae-a778-483911a1c1ce None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.293940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0446b92-8ad9-4eae-a778-483911a1c1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.301614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 906/1813] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:56.303158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f4859-b5e2-4979-95dc-11a75e5e39c0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.309670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-526f4859-b5e2-4979-95dc-11a75e5e39c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:56.323725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a0446b92-8ad9-4eae-a778-483911a1c1ce None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:56.325093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a0446b92-8ad9-4eae-a778-483911a1c1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:56.326775 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f4859-b5e2-4979-95dc-11a75e5e39c0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:56.395468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95079167-784c-401d-b91d-6f658fd995fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:56.396139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95079167-784c-401d-b91d-6f658fd995fe admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.396423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95079167-784c-401d-b91d-6f658fd995fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.396700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95079167-784c-401d-b91d-6f658fd995fe admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.397413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95079167-784c-401d-b91d-6f658fd995fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:56.399369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95079167-784c-401d-b91d-6f658fd995fe admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:56.404915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 908/1814] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:56.409034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0446b92-8ad9-4eae-a778-483911a1c1ce None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.435957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0446b92-8ad9-4eae-a778-483911a1c1ce None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:56.441786 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c276f3a-c7e2-428f-88b0-8267e297842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:56.442432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c276f3a-c7e2-428f-88b0-8267e297842c admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.442719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c276f3a-c7e2-428f-88b0-8267e297842c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.442999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c276f3a-c7e2-428f-88b0-8267e297842c admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.443839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c276f3a-c7e2-428f-88b0-8267e297842c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:56.447094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c276f3a-c7e2-428f-88b0-8267e297842c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:56.531231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 907/1815] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:56 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:56.532574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f5d7c-53dd-4007-a9f5-c77a79ba0fd1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.532890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f5d7c-53dd-4007-a9f5-c77a79ba0fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.533171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f5d7c-53dd-4007-a9f5-c77a79ba0fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.553879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e01f5d7c-53dd-4007-a9f5-c77a79ba0fd1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:56.556031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e01f5d7c-53dd-4007-a9f5-c77a79ba0fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:56.563366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 909/1816] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:56 2022] POST /identity/v3/projects => generated 409 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:56.563366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c276f3a-c7e2-428f-88b0-8267e297842c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.566980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 908/1817] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:56.568104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f5d7c-53dd-4007-a9f5-c77a79ba0fd1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.585277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f5d7c-53dd-4007-a9f5-c77a79ba0fd1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:56.655088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c276f3a-c7e2-428f-88b0-8267e297842c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:56.662885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aedf00-2abd-43b1-ad69-337a2b8b8fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:56.663692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aedf00-2abd-43b1-ad69-337a2b8b8fdf admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.664213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aedf00-2abd-43b1-ad69-337a2b8b8fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.665131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aedf00-2abd-43b1-ad69-337a2b8b8fdf admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.665950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aedf00-2abd-43b1-ad69-337a2b8b8fdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:56.668035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aedf00-2abd-43b1-ad69-337a2b8b8fdf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:56.669497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187ace5-7525-4da6-b28a-a3d4a2f336fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:56.670197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187ace5-7525-4da6-b28a-a3d4a2f336fb admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.670590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187ace5-7525-4da6-b28a-a3d4a2f336fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.670941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187ace5-7525-4da6-b28a-a3d4a2f336fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.671715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187ace5-7525-4da6-b28a-a3d4a2f336fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:56.673671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187ace5-7525-4da6-b28a-a3d4a2f336fb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:56.684957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 910/1818] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:56.685801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187ace5-7525-4da6-b28a-a3d4a2f336fb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.695468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187ace5-7525-4da6-b28a-a3d4a2f336fb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:56.700591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215b2e8-4c8b-4253-81c2-ceb4e100dd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:56.701187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215b2e8-4c8b-4253-81c2-ceb4e100dd51 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.701471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215b2e8-4c8b-4253-81c2-ceb4e100dd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.701748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215b2e8-4c8b-4253-81c2-ceb4e100dd51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.702430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215b2e8-4c8b-4253-81c2-ceb4e100dd51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:56.702735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215b2e8-4c8b-4253-81c2-ceb4e100dd51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:56.704348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215b2e8-4c8b-4253-81c2-ceb4e100dd51 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:56.716910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 911/1819] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:56.717808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215b2e8-4c8b-4253-81c2-ceb4e100dd51 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.719453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2215b2e8-4c8b-4253-81c2-ceb4e100dd51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:56.732765 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215b2e8-4c8b-4253-81c2-ceb4e100dd51 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:56.737437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:56.738063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.738349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.738634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] PATH_INFO: `/v3/projects/ac35b00fd43f4efd97f29731f9b02a19` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.739830 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac35b00fd43f4efd97f29731f9b02a19)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:56.741800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:56.799972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 909/1820] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:56 2022] POST /identity/v3/users => generated 305 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:56.800803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aedf00-2abd-43b1-ad69-337a2b8b8fdf admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.811015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aedf00-2abd-43b1-ad69-337a2b8b8fdf admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:56.815699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63073f94-e959-4f66-a862-2056bcce33a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:56.816249 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63073f94-e959-4f66-a862-2056bcce33a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.816529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63073f94-e959-4f66-a862-2056bcce33a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.816803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63073f94-e959-4f66-a862-2056bcce33a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.817574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63073f94-e959-4f66-a862-2056bcce33a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:56.819148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63073f94-e959-4f66-a862-2056bcce33a6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:56.828886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 910/1821] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:56.829620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63073f94-e959-4f66-a862-2056bcce33a6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.839124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63073f94-e959-4f66-a862-2056bcce33a6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:56.843739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7e2e9-fe76-43c5-bba7-f3fbc221214f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:56.844330 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7e2e9-fe76-43c5-bba7-f3fbc221214f admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:56.844615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7e2e9-fe76-43c5-bba7-f3fbc221214f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:56.844889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7e2e9-fe76-43c5-bba7-f3fbc221214f admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:56.845536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7e2e9-fe76-43c5-bba7-f3fbc221214f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:56.847349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7e2e9-fe76-43c5-bba7-f3fbc221214f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:56.864607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac35b00fd43f4efd97f29731f9b02a19', 'request_id': 'req-5ac3191a-b217-4fd1-8d00-f89af5db800e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.872130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac35b00fd43f4efd97f29731f9b02a19', 'request_id': 'req-5ac3191a-b217-4fd1-8d00-f89af5db800e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.936688 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 911/1822] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:56 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:56.936688 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7e2e9-fe76-43c5-bba7-f3fbc221214f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.953882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac35b00fd43f4efd97f29731f9b02a19', 'request_id': 'req-5ac3191a-b217-4fd1-8d00-f89af5db800e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.968260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] The token cache is being invalidate because project ac35b00fd43f4efd97f29731f9b02a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:22:56.968720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.969520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:22:56.971471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 912/1823] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:22:56 2022] DELETE /identity/v3/projects/ac35b00fd43f4efd97f29731f9b02a19 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:56.972085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:56.973496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:57.030592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7e2e9-fe76-43c5-bba7-f3fbc221214f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.035409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17edca1-cf34-4a26-9039-3f534f1028cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.036075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17edca1-cf34-4a26-9039-3f534f1028cf admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.036356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17edca1-cf34-4a26-9039-3f534f1028cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.036630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17edca1-cf34-4a26-9039-3f534f1028cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.037456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17edca1-cf34-4a26-9039-3f534f1028cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.039075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17edca1-cf34-4a26-9039-3f534f1028cf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.047068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3191a-b217-4fd1-8d00-f89af5db800e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.051568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 912/1824] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:57.051922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f491b69-9c62-45b7-986e-dbd3462f4cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.052631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f491b69-9c62-45b7-986e-dbd3462f4cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.052930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f491b69-9c62-45b7-986e-dbd3462f4cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.053188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f491b69-9c62-45b7-986e-dbd3462f4cf9 admin admin] PATH_INFO: `/v3/projects/6585ae3af94e4984aa2a9c4b33bd266a/users/da550a1e87a148baa154dd5e0b03b11d/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.056011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17edca1-cf34-4a26-9039-3f534f1028cf admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.057902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b17edca1-cf34-4a26-9039-3f534f1028cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:57.071502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f491b69-9c62-45b7-986e-dbd3462f4cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6585ae3af94e4984aa2a9c4b33bd266a, user_id=da550a1e87a148baa154dd5e0b03b11d, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.071867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17edca1-cf34-4a26-9039-3f534f1028cf admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.074379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f491b69-9c62-45b7-986e-dbd3462f4cf9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.078354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2c09c-01aa-4cde-b2f5-86bfde586c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.078981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2c09c-01aa-4cde-b2f5-86bfde586c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.079298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2c09c-01aa-4cde-b2f5-86bfde586c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.079536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2c09c-01aa-4cde-b2f5-86bfde586c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.080227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2c09c-01aa-4cde-b2f5-86bfde586c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.082119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2c09c-01aa-4cde-b2f5-86bfde586c39 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.162552 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 913/1825] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/users => generated 301 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:57.172434 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 913/1826] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:56 2022] PUT /identity/v3/projects/6585ae3af94e4984aa2a9c4b33bd266a/users/da550a1e87a148baa154dd5e0b03b11d/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:57.172602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fba16-1327-4fd5-b679-5404a4798867 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.172602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fba16-1327-4fd5-b679-5404a4798867 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.172602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fba16-1327-4fd5-b679-5404a4798867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.172807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f491b69-9c62-45b7-986e-dbd3462f4cf9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.174526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4f491b69-9c62-45b7-986e-dbd3462f4cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:57.192008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-269fba16-1327-4fd5-b679-5404a4798867 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:57.193254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-269fba16-1327-4fd5-b679-5404a4798867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:57.255245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f491b69-9c62-45b7-986e-dbd3462f4cf9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.260345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd8c48-27e8-4128-b27f-a91bc0903a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.261163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd8c48-27e8-4128-b27f-a91bc0903a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.261452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd8c48-27e8-4128-b27f-a91bc0903a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.261731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd8c48-27e8-4128-b27f-a91bc0903a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.262427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd8c48-27e8-4128-b27f-a91bc0903a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.264050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd8c48-27e8-4128-b27f-a91bc0903a33 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.269533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 914/1827] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:57.270600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fba16-1327-4fd5-b679-5404a4798867 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.277936 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 914/1828] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:57.279309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633261d-2221-42ad-b9de-da48229667ef None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.279673 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633261d-2221-42ad-b9de-da48229667ef None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.280071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633261d-2221-42ad-b9de-da48229667ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.280501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fba16-1327-4fd5-b679-5404a4798867 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.285606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac682f8-f179-47e1-a6a8-4cbf06022955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.286452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac682f8-f179-47e1-a6a8-4cbf06022955 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.286934 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac682f8-f179-47e1-a6a8-4cbf06022955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.287222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac682f8-f179-47e1-a6a8-4cbf06022955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.289050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:57.290090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac682f8-f179-47e1-a6a8-4cbf06022955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.291770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac682f8-f179-47e1-a6a8-4cbf06022955 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.301655 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 915/1829] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:57.302852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82d2a-9ee4-4bac-8a1e-83d2f98c1f59 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.303139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82d2a-9ee4-4bac-8a1e-83d2f98c1f59 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.303416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e82d2a-9ee4-4bac-8a1e-83d2f98c1f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.310173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3633261d-2221-42ad-b9de-da48229667ef None None] MFA Rules not processed for user `da550a1e87a148baa154dd5e0b03b11d`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:57.311539 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3633261d-2221-42ad-b9de-da48229667ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:57.323876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-39e82d2a-9ee4-4bac-8a1e-83d2f98c1f59 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:57.323876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-39e82d2a-9ee4-4bac-8a1e-83d2f98c1f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:57.331937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 916/1830] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:57.336496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e82d2a-9ee4-4bac-8a1e-83d2f98c1f59 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.348043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e82d2a-9ee4-4bac-8a1e-83d2f98c1f59 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.354957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04d910-2b52-4542-9904-95c3aa4c5e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.355637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04d910-2b52-4542-9904-95c3aa4c5e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.355959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04d910-2b52-4542-9904-95c3aa4c5e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.356242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04d910-2b52-4542-9904-95c3aa4c5e8a admin admin] PATH_INFO: `/v3/projects/2a934f171be14c0dbaafe8c4dfd3d97c/users/7875ed3841914aeaa1f76b21b0fab66c/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.372315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04d910-2b52-4542-9904-95c3aa4c5e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a934f171be14c0dbaafe8c4dfd3d97c, user_id=7875ed3841914aeaa1f76b21b0fab66c, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.378800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04d910-2b52-4542-9904-95c3aa4c5e8a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.383818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 915/1831] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:57.384712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633261d-2221-42ad-b9de-da48229667ef None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.395132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633261d-2221-42ad-b9de-da48229667ef None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.399836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee628089-17d3-4fc3-aaff-a1837392f733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.400443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee628089-17d3-4fc3-aaff-a1837392f733 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.400719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee628089-17d3-4fc3-aaff-a1837392f733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.400994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee628089-17d3-4fc3-aaff-a1837392f733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.401689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee628089-17d3-4fc3-aaff-a1837392f733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.403359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee628089-17d3-4fc3-aaff-a1837392f733 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.413280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 916/1832] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:57.414078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee628089-17d3-4fc3-aaff-a1837392f733 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.425512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee628089-17d3-4fc3-aaff-a1837392f733 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.434740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ec09-8a32-494f-b997-86aaa0450355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.434740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ec09-8a32-494f-b997-86aaa0450355 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.434740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ec09-8a32-494f-b997-86aaa0450355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.434740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ec09-8a32-494f-b997-86aaa0450355 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.435085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ec09-8a32-494f-b997-86aaa0450355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:22:57.435085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ec09-8a32-494f-b997-86aaa0450355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.435581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ec09-8a32-494f-b997-86aaa0450355 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.448078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 917/1833] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:22:57.448983 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ec09-8a32-494f-b997-86aaa0450355 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.452883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-53c6ec09-8a32-494f-b997-86aaa0450355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:57.459851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 917/1834] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:57 2022] PUT /identity/v3/projects/2a934f171be14c0dbaafe8c4dfd3d97c/users/7875ed3841914aeaa1f76b21b0fab66c/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:57.460976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04d910-2b52-4542-9904-95c3aa4c5e8a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.465048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:57.530075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04d910-2b52-4542-9904-95c3aa4c5e8a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.534742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f66e2-2171-46c3-a3b9-991ef7c0f4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.535412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f66e2-2171-46c3-a3b9-991ef7c0f4db admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.535693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f66e2-2171-46c3-a3b9-991ef7c0f4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.537082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f66e2-2171-46c3-a3b9-991ef7c0f4db admin admin] PATH_INFO: `/v3/projects/b2376704712f4764a383346953b7ee63/users/63b2945cd5424af8b411198e66d03f1e/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.548905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ec09-8a32-494f-b997-86aaa0450355 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.551094 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f66e2-2171-46c3-a3b9-991ef7c0f4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2376704712f4764a383346953b7ee63, user_id=63b2945cd5424af8b411198e66d03f1e, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.553840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f66e2-2171-46c3-a3b9-991ef7c0f4db admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.598265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f07ca3-1700-4f07-82a0-a7366cd02de4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a234a96eccd4f38b0213d7d0461ad24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.598887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07ca3-1700-4f07-82a0-a7366cd02de4 service glance] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.599167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07ca3-1700-4f07-82a0-a7366cd02de4 service glance] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.599437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07ca3-1700-4f07-82a0-a7366cd02de4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.616590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 918/1835] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:57 2022] PUT /identity/v3/projects/b2376704712f4764a383346953b7ee63/users/63b2945cd5424af8b411198e66d03f1e/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:57.626677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f66e2-2171-46c3-a3b9-991ef7c0f4db admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.656050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f07ca3-1700-4f07-82a0-a7366cd02de4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.657627 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f07ca3-1700-4f07-82a0-a7366cd02de4 service glance] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.666390 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 918/1836] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:57.667656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f11fb-7e8a-4490-ad37-713c829b8bb5 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.667971 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f11fb-7e8a-4490-ad37-713c829b8bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.668256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f11fb-7e8a-4490-ad37-713c829b8bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.692992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f66e2-2171-46c3-a3b9-991ef7c0f4db admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.697432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c49f11fb-7e8a-4490-ad37-713c829b8bb5 None None] MFA Rules not processed for user `7875ed3841914aeaa1f76b21b0fab66c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:57.698650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c49f11fb-7e8a-4490-ad37-713c829b8bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:57.698753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dedebe-d416-4f49-97da-a149747f4219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.699625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dedebe-d416-4f49-97da-a149747f4219 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.699953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dedebe-d416-4f49-97da-a149747f4219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.700233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dedebe-d416-4f49-97da-a149747f4219 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.700938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dedebe-d416-4f49-97da-a149747f4219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.702867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dedebe-d416-4f49-97da-a149747f4219 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.764886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 919/1837] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:57.766118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab546250-1497-4a0e-95d8-afe617f29e5b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.766459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab546250-1497-4a0e-95d8-afe617f29e5b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.766738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab546250-1497-4a0e-95d8-afe617f29e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.788731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 919/1838] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:57.788731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dedebe-d416-4f49-97da-a149747f4219 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.798355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ab546250-1497-4a0e-95d8-afe617f29e5b None None] MFA Rules not processed for user `63b2945cd5424af8b411198e66d03f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:57.799608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ab546250-1497-4a0e-95d8-afe617f29e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:57.861063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dedebe-d416-4f49-97da-a149747f4219 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.866046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9311c-6020-4fe6-a3a3-ad1b517ba7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.866673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9311c-6020-4fe6-a3a3-ad1b517ba7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.867151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9311c-6020-4fe6-a3a3-ad1b517ba7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.867252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9311c-6020-4fe6-a3a3-ad1b517ba7d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.868666 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9311c-6020-4fe6-a3a3-ad1b517ba7d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.870564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9311c-6020-4fe6-a3a3-ad1b517ba7d5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.871105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 920/1839] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:57.881751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab546250-1497-4a0e-95d8-afe617f29e5b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.888370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab546250-1497-4a0e-95d8-afe617f29e5b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:57.893127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a1f78-445a-45b6-903e-f38d21aa9c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:57.893760 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a1f78-445a-45b6-903e-f38d21aa9c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:57.894196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a1f78-445a-45b6-903e-f38d21aa9c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:57.894483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a1f78-445a-45b6-903e-f38d21aa9c60 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:57.895211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a1f78-445a-45b6-903e-f38d21aa9c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:57.897214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a1f78-445a-45b6-903e-f38d21aa9c60 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:57.952404 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 920/1840] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/projects => generated 407 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:57.953464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9311c-6020-4fe6-a3a3-ad1b517ba7d5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.980754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 921/1841] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:57.981687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a1f78-445a-45b6-903e-f38d21aa9c60 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:57.992152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a1f78-445a-45b6-903e-f38d21aa9c60 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.020995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9311c-6020-4fe6-a3a3-ad1b517ba7d5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.026564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920fa34-b586-4c7a-beeb-0b023f269e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.027224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920fa34-b586-4c7a-beeb-0b023f269e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.027672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920fa34-b586-4c7a-beeb-0b023f269e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.028022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920fa34-b586-4c7a-beeb-0b023f269e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.028670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920fa34-b586-4c7a-beeb-0b023f269e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.030649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920fa34-b586-4c7a-beeb-0b023f269e55 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.049984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632bed5-6ced-42cc-bf89-0bdd1cc761d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.050632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632bed5-6ced-42cc-bf89-0bdd1cc761d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.050914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632bed5-6ced-42cc-bf89-0bdd1cc761d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.051245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632bed5-6ced-42cc-bf89-0bdd1cc761d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.051996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632bed5-6ced-42cc-bf89-0bdd1cc761d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.054008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632bed5-6ced-42cc-bf89-0bdd1cc761d4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.140603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 921/1842] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/projects => generated 399 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:58.141456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920fa34-b586-4c7a-beeb-0b023f269e55 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.177629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 922/1843] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:57 2022] POST /identity/v3/users => generated 310 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:58.178792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632bed5-6ced-42cc-bf89-0bdd1cc761d4 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.189781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632bed5-6ced-42cc-bf89-0bdd1cc761d4 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.218297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920fa34-b586-4c7a-beeb-0b023f269e55 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.223039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed40a2-6e15-45c4-8070-55c1f923495f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.223662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed40a2-6e15-45c4-8070-55c1f923495f admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.223984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed40a2-6e15-45c4-8070-55c1f923495f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.224263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed40a2-6e15-45c4-8070-55c1f923495f admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.224961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed40a2-6e15-45c4-8070-55c1f923495f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.226541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed40a2-6e15-45c4-8070-55c1f923495f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.237405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 922/1844] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:58.238290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed40a2-6e15-45c4-8070-55c1f923495f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.249015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed40a2-6e15-45c4-8070-55c1f923495f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.255682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729ccc-056b-4285-957b-e341b1446f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.256538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729ccc-056b-4285-957b-e341b1446f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.256962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729ccc-056b-4285-957b-e341b1446f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.257394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729ccc-056b-4285-957b-e341b1446f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.258153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28729ccc-056b-4285-957b-e341b1446f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.259209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b67a4e-b323-4315-8ad2-633139964e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.259926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28729ccc-056b-4285-957b-e341b1446f5a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.260004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b67a4e-b323-4315-8ad2-633139964e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.260148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b67a4e-b323-4315-8ad2-633139964e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.260397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b67a4e-b323-4315-8ad2-633139964e81 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.261216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b67a4e-b323-4315-8ad2-633139964e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.263283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b67a4e-b323-4315-8ad2-633139964e81 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.271658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 923/1845] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:58.272524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729ccc-056b-4285-957b-e341b1446f5a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.283280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729ccc-056b-4285-957b-e341b1446f5a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.288907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81b838-6b2f-4e5a-9e2d-10bf82feb6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.290274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b838-6b2f-4e5a-9e2d-10bf82feb6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.290558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b838-6b2f-4e5a-9e2d-10bf82feb6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.290837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b838-6b2f-4e5a-9e2d-10bf82feb6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.291941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81b838-6b2f-4e5a-9e2d-10bf82feb6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.293920 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81b838-6b2f-4e5a-9e2d-10bf82feb6c7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.311917 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 924/1846] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:58.311917 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81b838-6b2f-4e5a-9e2d-10bf82feb6c7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.322420 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81b838-6b2f-4e5a-9e2d-10bf82feb6c7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.327142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b324b7d-517d-455f-b867-2e5dd97704f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.327943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b324b7d-517d-455f-b867-2e5dd97704f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.328226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b324b7d-517d-455f-b867-2e5dd97704f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.328498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b324b7d-517d-455f-b867-2e5dd97704f9 admin admin] PATH_INFO: `/v3/projects/b0da46341b2242719351e6c49c6b26b0/users/935847942dbf491fafeae65d24397b18/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.344496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b324b7d-517d-455f-b867-2e5dd97704f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0da46341b2242719351e6c49c6b26b0, user_id=935847942dbf491fafeae65d24397b18, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.347201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b324b7d-517d-455f-b867-2e5dd97704f9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.347606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 923/1847] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:58 2022] POST /identity/v3/users => generated 306 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:58.348920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b67a4e-b323-4315-8ad2-633139964e81 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.360995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b67a4e-b323-4315-8ad2-633139964e81 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.366732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ca2aa-dc1d-428d-ad83-44002eee9ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.367974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ca2aa-dc1d-428d-ad83-44002eee9ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.367974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ca2aa-dc1d-428d-ad83-44002eee9ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.368136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ca2aa-dc1d-428d-ad83-44002eee9ed2 admin admin] PATH_INFO: `/v3/projects/b23040297084451e938b4274f102f4dd/users/d2f3e4b8c5b246f58e3011976d893369/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.383352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ca2aa-dc1d-428d-ad83-44002eee9ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23040297084451e938b4274f102f4dd, user_id=d2f3e4b8c5b246f58e3011976d893369, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.386037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ca2aa-dc1d-428d-ad83-44002eee9ed2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.450908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 925/1848] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:58 2022] PUT /identity/v3/projects/b0da46341b2242719351e6c49c6b26b0/users/935847942dbf491fafeae65d24397b18/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:58.451752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b324b7d-517d-455f-b867-2e5dd97704f9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.462713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 924/1849] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:58 2022] PUT /identity/v3/projects/b23040297084451e938b4274f102f4dd/users/d2f3e4b8c5b246f58e3011976d893369/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:58.463729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ca2aa-dc1d-428d-ad83-44002eee9ed2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.513660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b324b7d-517d-455f-b867-2e5dd97704f9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.526723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ca2aa-dc1d-428d-ad83-44002eee9ed2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.531971 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cefbd-b5a8-41d3-962e-f5bf1d53a298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.532665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cefbd-b5a8-41d3-962e-f5bf1d53a298 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.533086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cefbd-b5a8-41d3-962e-f5bf1d53a298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.533425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cefbd-b5a8-41d3-962e-f5bf1d53a298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.534121 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cefbd-b5a8-41d3-962e-f5bf1d53a298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.535818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cefbd-b5a8-41d3-962e-f5bf1d53a298 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.546752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:58.547539 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 925/1850] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:58.548660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20348caa-62a2-4ced-a321-45105a43aed5 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.548947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20348caa-62a2-4ced-a321-45105a43aed5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.549224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20348caa-62a2-4ced-a321-45105a43aed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.582288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-20348caa-62a2-4ced-a321-45105a43aed5 None None] MFA Rules not processed for user `d2f3e4b8c5b246f58e3011976d893369`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:58.583839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-20348caa-62a2-4ced-a321-45105a43aed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:58.610825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb073288-b8ca-4f3f-8f2d-9e48aa3ba032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.610825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb073288-b8ca-4f3f-8f2d-9e48aa3ba032 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.610825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb073288-b8ca-4f3f-8f2d-9e48aa3ba032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.610825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb073288-b8ca-4f3f-8f2d-9e48aa3ba032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.611232 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb073288-b8ca-4f3f-8f2d-9e48aa3ba032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.612035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb073288-b8ca-4f3f-8f2d-9e48aa3ba032 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.628658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 926/1851] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:58.628658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb073288-b8ca-4f3f-8f2d-9e48aa3ba032 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.642030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb073288-b8ca-4f3f-8f2d-9e48aa3ba032 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.647294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc201bff-735c-429a-bdb7-29eaccd64c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.648012 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc201bff-735c-429a-bdb7-29eaccd64c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.648529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc201bff-735c-429a-bdb7-29eaccd64c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.649354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc201bff-735c-429a-bdb7-29eaccd64c8f admin admin] PATH_INFO: `/v3/projects/b0da46341b2242719351e6c49c6b26b0/users/935847942dbf491fafeae65d24397b18/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.651642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc201bff-735c-429a-bdb7-29eaccd64c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0da46341b2242719351e6c49c6b26b0, user_id=935847942dbf491fafeae65d24397b18, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.654685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc201bff-735c-429a-bdb7-29eaccd64c8f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.667755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 926/1852] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:58.668893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20348caa-62a2-4ced-a321-45105a43aed5 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.671942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 927/1853] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:58 2022] PUT /identity/v3/projects/b0da46341b2242719351e6c49c6b26b0/users/935847942dbf491fafeae65d24397b18/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:58.673439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc201bff-735c-429a-bdb7-29eaccd64c8f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.689293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20348caa-62a2-4ced-a321-45105a43aed5 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.748323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc201bff-735c-429a-bdb7-29eaccd64c8f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.752360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa12fa-ddab-472f-99bb-ab47eab75051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.753119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa12fa-ddab-472f-99bb-ab47eab75051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.753367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecdd3d-0004-49b5-b536-560fc5c798a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.753478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa12fa-ddab-472f-99bb-ab47eab75051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.753671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa12fa-ddab-472f-99bb-ab47eab75051 admin admin] PATH_INFO: `/v3/projects/dce806ea37f2468790b5138b37aa6e5e/users/a4b078f6602c484bbb3e5691d77e632a/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.754018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecdd3d-0004-49b5-b536-560fc5c798a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.754278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecdd3d-0004-49b5-b536-560fc5c798a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.754561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecdd3d-0004-49b5-b536-560fc5c798a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.755244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecdd3d-0004-49b5-b536-560fc5c798a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.757237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecdd3d-0004-49b5-b536-560fc5c798a1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.768457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa12fa-ddab-472f-99bb-ab47eab75051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce806ea37f2468790b5138b37aa6e5e, user_id=a4b078f6602c484bbb3e5691d77e632a, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.771240 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa12fa-ddab-472f-99bb-ab47eab75051 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.916442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 927/1854] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:58 2022] PUT /identity/v3/projects/dce806ea37f2468790b5138b37aa6e5e/users/a4b078f6602c484bbb3e5691d77e632a/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:58.920677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 928/1855] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:22:58 2022] POST /identity/v3/projects => generated 381 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:58.920899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6449774-b02e-4b24-be67-9fa16e71f7ee None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.920899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6449774-b02e-4b24-be67-9fa16e71f7ee None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.920899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6449774-b02e-4b24-be67-9fa16e71f7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.921133 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecdd3d-0004-49b5-b536-560fc5c798a1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:58.981265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecdd3d-0004-49b5-b536-560fc5c798a1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:58.986095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdd4c6-4acb-4ee3-853f-4c9a13ac3fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:58.986717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd4c6-4acb-4ee3-853f-4c9a13ac3fab admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:58.987003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd4c6-4acb-4ee3-853f-4c9a13ac3fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:58.987281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdd4c6-4acb-4ee3-853f-4c9a13ac3fab admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:58.988000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdd4c6-4acb-4ee3-853f-4c9a13ac3fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:58.989699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdd4c6-4acb-4ee3-853f-4c9a13ac3fab admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:58.999761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 929/1856] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:22:59.000536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdd4c6-4acb-4ee3-853f-4c9a13ac3fab admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.007876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:22:59.010302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdd4c6-4acb-4ee3-853f-4c9a13ac3fab admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:59.015274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe8bcd-77a3-4f1f-be8c-2b64becd8e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:59.015907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe8bcd-77a3-4f1f-be8c-2b64becd8e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.017779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d6449774-b02e-4b24-be67-9fa16e71f7ee None None] MFA Rules not processed for user `935847942dbf491fafeae65d24397b18`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:59.017879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe8bcd-77a3-4f1f-be8c-2b64becd8e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.017879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe8bcd-77a3-4f1f-be8c-2b64becd8e9d admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.017879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe8bcd-77a3-4f1f-be8c-2b64becd8e9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:59.018091 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d6449774-b02e-4b24-be67-9fa16e71f7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:59.019131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe8bcd-77a3-4f1f-be8c-2b64becd8e9d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:59.083601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 928/1857] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:58 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:59.084447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6449774-b02e-4b24-be67-9fa16e71f7ee None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.095170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6449774-b02e-4b24-be67-9fa16e71f7ee None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:59.099927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc490a-c49f-456f-b09f-a12a2ae6ba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:59.100578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc490a-c49f-456f-b09f-a12a2ae6ba57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.100857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc490a-c49f-456f-b09f-a12a2ae6ba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.101129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc490a-c49f-456f-b09f-a12a2ae6ba57 admin admin] PATH_INFO: `/v3/projects/dce806ea37f2468790b5138b37aa6e5e/users/a4b078f6602c484bbb3e5691d77e632a/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.103036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc490a-c49f-456f-b09f-a12a2ae6ba57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce806ea37f2468790b5138b37aa6e5e, user_id=a4b078f6602c484bbb3e5691d77e632a, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:59.106062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc490a-c49f-456f-b09f-a12a2ae6ba57 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:59.115952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 929/1858] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:59 2022] PUT /identity/v3/projects/dce806ea37f2468790b5138b37aa6e5e/users/a4b078f6602c484bbb3e5691d77e632a/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:59.116781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc490a-c49f-456f-b09f-a12a2ae6ba57 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.122604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-40dc490a-c49f-456f-b09f-a12a2ae6ba57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:59.139752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 930/1859] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:22:58 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:22:59.141036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90033c-63e4-4d95-855d-e2f7e0807e89 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.141326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90033c-63e4-4d95-855d-e2f7e0807e89 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.141609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90033c-63e4-4d95-855d-e2f7e0807e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.170878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3e90033c-63e4-4d95-855d-e2f7e0807e89 None None] MFA Rules not processed for user `a4b078f6602c484bbb3e5691d77e632a`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:59.172037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3e90033c-63e4-4d95-855d-e2f7e0807e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:59.179505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc490a-c49f-456f-b09f-a12a2ae6ba57 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:59.184545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7d435-f740-4b98-a01d-73928db8f362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:59.185171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7d435-f740-4b98-a01d-73928db8f362 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.185453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7d435-f740-4b98-a01d-73928db8f362 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.185727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7d435-f740-4b98-a01d-73928db8f362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.250809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7d435-f740-4b98-a01d-73928db8f362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:59.252309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7d435-f740-4b98-a01d-73928db8f362 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:59.260116 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 931/1860] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:22:59.260116 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90033c-63e4-4d95-855d-e2f7e0807e89 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.266311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 930/1861] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:59 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:59.267656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7d435-f740-4b98-a01d-73928db8f362 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.269311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fcb7d435-f740-4b98-a01d-73928db8f362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:59.282354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7d435-f740-4b98-a01d-73928db8f362 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:59.287731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06745a-c79c-46a7-9528-1888bdbc461e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:59.288522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06745a-c79c-46a7-9528-1888bdbc461e service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.288908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06745a-c79c-46a7-9528-1888bdbc461e service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.289359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06745a-c79c-46a7-9528-1888bdbc461e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.302349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06745a-c79c-46a7-9528-1888bdbc461e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:59.303907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06745a-c79c-46a7-9528-1888bdbc461e service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:59.317648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 931/1862] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:59 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:22:59.326091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90033c-63e4-4d95-855d-e2f7e0807e89 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:59.334962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331db705-fce6-4aea-81eb-6ec7a55f604f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:59.335600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331db705-fce6-4aea-81eb-6ec7a55f604f admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.335916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331db705-fce6-4aea-81eb-6ec7a55f604f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.336196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331db705-fce6-4aea-81eb-6ec7a55f604f admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.337538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331db705-fce6-4aea-81eb-6ec7a55f604f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:59.339139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331db705-fce6-4aea-81eb-6ec7a55f604f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:59.356677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 932/1863] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:59.357861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06745a-c79c-46a7-9528-1888bdbc461e service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.373502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06745a-c79c-46a7-9528-1888bdbc461e service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:59.378200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa84836-d6d4-4864-bf7b-ee7710890f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:59.378871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa84836-d6d4-4864-bf7b-ee7710890f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.379152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa84836-d6d4-4864-bf7b-ee7710890f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.379435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa84836-d6d4-4864-bf7b-ee7710890f5d admin admin] PATH_INFO: `/v3/projects/9e488e63ff33495ab626b4393b61675e/users/f9e04f89491548149d7cd8223c6d292c/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.399851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa84836-d6d4-4864-bf7b-ee7710890f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e488e63ff33495ab626b4393b61675e, user_id=f9e04f89491548149d7cd8223c6d292c, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:59.402918 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa84836-d6d4-4864-bf7b-ee7710890f5d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:59.464520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 932/1864] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:59 2022] PUT /identity/v3/projects/9e488e63ff33495ab626b4393b61675e/users/f9e04f89491548149d7cd8223c6d292c/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:22:59.468718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331db705-fce6-4aea-81eb-6ec7a55f604f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.479284 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331db705-fce6-4aea-81eb-6ec7a55f604f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:59.538114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2435b78-3dbc-4917-8397-dc7244c6dfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:59.538754 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2435b78-3dbc-4917-8397-dc7244c6dfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.539055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2435b78-3dbc-4917-8397-dc7244c6dfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.539310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2435b78-3dbc-4917-8397-dc7244c6dfc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.540981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2435b78-3dbc-4917-8397-dc7244c6dfc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:59.542412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2435b78-3dbc-4917-8397-dc7244c6dfc1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:59.558152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa84836-d6d4-4864-bf7b-ee7710890f5d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.562137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 933/1865] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:22:59.563734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2435b78-3dbc-4917-8397-dc7244c6dfc1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.573989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2435b78-3dbc-4917-8397-dc7244c6dfc1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:59.582062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ede57-9719-45e1-9d3c-235f216f428d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:59.582755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ede57-9719-45e1-9d3c-235f216f428d admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.583044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ede57-9719-45e1-9d3c-235f216f428d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.583331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ede57-9719-45e1-9d3c-235f216f428d admin admin] PATH_INFO: `/v3/projects/9e488e63ff33495ab626b4393b61675e/users/f9e04f89491548149d7cd8223c6d292c/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.586079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ede57-9719-45e1-9d3c-235f216f428d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e488e63ff33495ab626b4393b61675e, user_id=f9e04f89491548149d7cd8223c6d292c, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:59.589821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ede57-9719-45e1-9d3c-235f216f428d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:59.605714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 934/1866] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:22:59 2022] PUT /identity/v3/projects/9e488e63ff33495ab626b4393b61675e/users/f9e04f89491548149d7cd8223c6d292c/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:22:59.616386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a74ca-a82a-4d5b-a3c3-331314a5b184 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.616683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a74ca-a82a-4d5b-a3c3-331314a5b184 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.616968 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a74ca-a82a-4d5b-a3c3-331314a5b184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.618571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa84836-d6d4-4864-bf7b-ee7710890f5d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:59.645985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-263a74ca-a82a-4d5b-a3c3-331314a5b184 None None] MFA Rules not processed for user `f9e04f89491548149d7cd8223c6d292c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:22:59.647196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-263a74ca-a82a-4d5b-a3c3-331314a5b184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:22:59.666373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee62a2c-6d86-4c57-8bf5-3ba81d0b13d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:59.667006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee62a2c-6d86-4c57-8bf5-3ba81d0b13d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.667293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee62a2c-6d86-4c57-8bf5-3ba81d0b13d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.667572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee62a2c-6d86-4c57-8bf5-3ba81d0b13d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.714061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 935/1867] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:22:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:22:59.721538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee62a2c-6d86-4c57-8bf5-3ba81d0b13d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:59.722971 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee62a2c-6d86-4c57-8bf5-3ba81d0b13d9 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:59.732657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 933/1868] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:59 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:59.823625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a74ca-a82a-4d5b-a3c3-331314a5b184 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.835198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a74ca-a82a-4d5b-a3c3-331314a5b184 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:22:59.842075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa01e38-886a-408e-8aa2-b4728044190c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:22:59.844656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa01e38-886a-408e-8aa2-b4728044190c service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:22:59.844656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa01e38-886a-408e-8aa2-b4728044190c service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:22:59.844656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa01e38-886a-408e-8aa2-b4728044190c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:22:59.853601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa01e38-886a-408e-8aa2-b4728044190c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:22:59.855295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa01e38-886a-408e-8aa2-b4728044190c service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:22:59.866060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 936/1869] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:22:59.986706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee62a2c-6d86-4c57-8bf5-3ba81d0b13d9 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:22:59.997750 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee62a2c-6d86-4c57-8bf5-3ba81d0b13d9 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:00.001479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca45e2-8f64-4746-a228-632bd49a2586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:00.002094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca45e2-8f64-4746-a228-632bd49a2586 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:00.002378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca45e2-8f64-4746-a228-632bd49a2586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:00.002655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca45e2-8f64-4746-a228-632bd49a2586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:00.072940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca45e2-8f64-4746-a228-632bd49a2586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:00.074611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca45e2-8f64-4746-a228-632bd49a2586 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:00.085629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 934/1870] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:22:59 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:00.131461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:00.499255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:01.128452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa01e38-886a-408e-8aa2-b4728044190c service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:01.190531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa01e38-886a-408e-8aa2-b4728044190c service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:01.195120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:01.195795 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:01.196091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:01.196369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] PATH_INFO: `/v3/users/d2f3e4b8c5b246f58e3011976d893369` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:01.197516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f3e4b8c5b246f58e3011976d893369)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:01.199410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:01.737878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f3e4b8c5b246f58e3011976d893369'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:01.745255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f3e4b8c5b246f58e3011976d893369'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:01.815893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f3e4b8c5b246f58e3011976d893369'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:01.816512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f3e4b8c5b246f58e3011976d893369'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:01.828654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f3e4b8c5b246f58e3011976d893369'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:01.834655 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 937/1871] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:01 2022] DELETE /identity/v3/users/d2f3e4b8c5b246f58e3011976d893369 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:01.839284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9cba-b426-4eca-a6e6-e4fe09d44173 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:01.839718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9cba-b426-4eca-a6e6-e4fe09d44173 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:01.839718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9cba-b426-4eca-a6e6-e4fe09d44173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:01.859564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-1d5b9cba-b426-4eca-a6e6-e4fe09d44173 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:01.860892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1d5b9cba-b426-4eca-a6e6-e4fe09d44173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:01.926729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 935/1872] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:01.937778 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:01.939285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:01.997678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347844c6-194f-408d-9ea9-9cb635f6ca20 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:02.001620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a9990-220c-4628-b13c-e35a48896e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:02.002248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a9990-220c-4628-b13c-e35a48896e09 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:02.002531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a9990-220c-4628-b13c-e35a48896e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:02.002805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a9990-220c-4628-b13c-e35a48896e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:02.013160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a9990-220c-4628-b13c-e35a48896e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:02.014630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a9990-220c-4628-b13c-e35a48896e09 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:02.025871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 938/1873] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:02.476856 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b9cba-b426-4eca-a6e6-e4fe09d44173 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:02.479255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1d5b9cba-b426-4eca-a6e6-e4fe09d44173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:02.491438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b9cba-b426-4eca-a6e6-e4fe09d44173 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:02.496123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:02.498363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:02.498363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:02.498363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] PATH_INFO: `/v3/projects/b23040297084451e938b4274f102f4dd` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:02.498626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23040297084451e938b4274f102f4dd)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:02.500407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:02.613445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23040297084451e938b4274f102f4dd', 'request_id': 'req-9becc320-e231-4ca5-92dc-80b1a7648a6a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:02.619411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23040297084451e938b4274f102f4dd', 'request_id': 'req-9becc320-e231-4ca5-92dc-80b1a7648a6a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:02.624439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:02.704527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23040297084451e938b4274f102f4dd', 'request_id': 'req-9becc320-e231-4ca5-92dc-80b1a7648a6a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:02.716469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] The token cache is being invalidate because project b23040297084451e938b4274f102f4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:02.716783 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:02.717414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:02.718921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 936/1874] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:02 2022] DELETE /identity/v3/projects/b23040297084451e938b4274f102f4dd => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:02.722991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a9990-220c-4628-b13c-e35a48896e09 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:02.733692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a9990-220c-4628-b13c-e35a48896e09 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:02.792226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:02.792862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:02.793144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:02.793418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] PATH_INFO: `/v3/users/f9e04f89491548149d7cd8223c6d292c` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:02.794556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9e04f89491548149d7cd8223c6d292c)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:02.796631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:02.919356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:02.960300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9e04f89491548149d7cd8223c6d292c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:02.967547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9e04f89491548149d7cd8223c6d292c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:03.061674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9e04f89491548149d7cd8223c6d292c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:03.062394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9e04f89491548149d7cd8223c6d292c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:03.076578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-42692bda-a961-40e4-8321-2b59915af30d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9e04f89491548149d7cd8223c6d292c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:03.085505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 939/1875] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:02 2022] DELETE /identity/v3/users/f9e04f89491548149d7cd8223c6d292c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:03.979198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:03.985584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:04.006394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becc320-e231-4ca5-92dc-80b1a7648a6a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:04.078442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:04.079637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:04.079959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:04.080352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] PATH_INFO: `/v3/projects/9e488e63ff33495ab626b4393b61675e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:04.081679 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e488e63ff33495ab626b4393b61675e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:04.083629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:04.203215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:04.228307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e488e63ff33495ab626b4393b61675e', 'request_id': 'req-99fbbb5f-395a-45ab-8d97-eab268aba6b3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:04.238001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e488e63ff33495ab626b4393b61675e', 'request_id': 'req-99fbbb5f-395a-45ab-8d97-eab268aba6b3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:04.305374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e488e63ff33495ab626b4393b61675e', 'request_id': 'req-99fbbb5f-395a-45ab-8d97-eab268aba6b3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:04.319562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] The token cache is being invalidate because project 9e488e63ff33495ab626b4393b61675e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:04.319879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:04.320562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-99fbbb5f-395a-45ab-8d97-eab268aba6b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:04.322835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 937/1876] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:03 2022] DELETE /identity/v3/projects/9e488e63ff33495ab626b4393b61675e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:04.327900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42453e4b-2cfe-4f36-95e5-ac0886d57b65 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:04.328478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42453e4b-2cfe-4f36-95e5-ac0886d57b65 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:04.328937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42453e4b-2cfe-4f36-95e5-ac0886d57b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:04.353210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-42453e4b-2cfe-4f36-95e5-ac0886d57b65 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:04.355067 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-42453e4b-2cfe-4f36-95e5-ac0886d57b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:04.424143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 940/1877] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:04.434820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c35c9-077c-4de4-a29e-e60521413827 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:04.435101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c35c9-077c-4de4-a29e-e60521413827 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:04.435380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c35c9-077c-4de4-a29e-e60521413827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:04.456399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-927c35c9-077c-4de4-a29e-e60521413827 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:04.457922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-927c35c9-077c-4de4-a29e-e60521413827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:04.467331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 938/1878] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:04.471597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42453e4b-2cfe-4f36-95e5-ac0886d57b65 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:04.482061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42453e4b-2cfe-4f36-95e5-ac0886d57b65 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:04.486885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae1059-6846-45ee-b02e-ce9ede4c59d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:04.487502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae1059-6846-45ee-b02e-ce9ede4c59d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:04.487799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae1059-6846-45ee-b02e-ce9ede4c59d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:04.488095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae1059-6846-45ee-b02e-ce9ede4c59d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:04.488792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae1059-6846-45ee-b02e-ce9ede4c59d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:04.489101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae1059-6846-45ee-b02e-ce9ede4c59d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:04.490674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae1059-6846-45ee-b02e-ce9ede4c59d4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:04.503656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 941/1879] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:04.510266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c35c9-077c-4de4-a29e-e60521413827 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:04.521305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c35c9-077c-4de4-a29e-e60521413827 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:04.532244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597cecd-fde6-4e55-b236-ad54a499312d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:04.532244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597cecd-fde6-4e55-b236-ad54a499312d admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:04.532244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597cecd-fde6-4e55-b236-ad54a499312d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:04.532244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597cecd-fde6-4e55-b236-ad54a499312d admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:04.532572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597cecd-fde6-4e55-b236-ad54a499312d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:04.532572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597cecd-fde6-4e55-b236-ad54a499312d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:04.687904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 939/1880] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:04 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:04.693807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae1059-6846-45ee-b02e-ce9ede4c59d4 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:04.704711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae1059-6846-45ee-b02e-ce9ede4c59d4 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:04.727757 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:04.763502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb1b69-5cde-4aa1-9053-ad1fed8c78d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:04.764165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb1b69-5cde-4aa1-9053-ad1fed8c78d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:04.764458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb1b69-5cde-4aa1-9053-ad1fed8c78d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:04.764738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb1b69-5cde-4aa1-9053-ad1fed8c78d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:04.765375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb1b69-5cde-4aa1-9053-ad1fed8c78d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:04.767321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb1b69-5cde-4aa1-9053-ad1fed8c78d9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:04.941683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 942/1881] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:04 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:04.944471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597cecd-fde6-4e55-b236-ad54a499312d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:04.960701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597cecd-fde6-4e55-b236-ad54a499312d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:04.976397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2755571-45bf-4863-86d8-6f58c26e6297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:04.977262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2755571-45bf-4863-86d8-6f58c26e6297 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:04.977844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2755571-45bf-4863-86d8-6f58c26e6297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:04.978323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2755571-45bf-4863-86d8-6f58c26e6297 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:04.979868 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2755571-45bf-4863-86d8-6f58c26e6297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:04.982232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2755571-45bf-4863-86d8-6f58c26e6297 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:04.999586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 940/1882] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:05.000540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb1b69-5cde-4aa1-9053-ad1fed8c78d9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:05.012876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb1b69-5cde-4aa1-9053-ad1fed8c78d9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:05.020958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9baa5ec-8685-412c-ba3a-fdfce1ad0f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:05.021604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baa5ec-8685-412c-ba3a-fdfce1ad0f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:05.021881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baa5ec-8685-412c-ba3a-fdfce1ad0f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:05.022157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9baa5ec-8685-412c-ba3a-fdfce1ad0f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:05.022863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9baa5ec-8685-412c-ba3a-fdfce1ad0f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:05.025111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9baa5ec-8685-412c-ba3a-fdfce1ad0f87 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:05.036554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 943/1883] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:05.040833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2755571-45bf-4863-86d8-6f58c26e6297 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:05.050790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2755571-45bf-4863-86d8-6f58c26e6297 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:05.055334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5f53e-9ea2-46ed-9774-d971fc1a100a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:05.056053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5f53e-9ea2-46ed-9774-d971fc1a100a admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:05.056338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5f53e-9ea2-46ed-9774-d971fc1a100a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:05.056623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5f53e-9ea2-46ed-9774-d971fc1a100a admin admin] PATH_INFO: `/v3/projects/87ebf95cc93b48648fd9db91b5cfdcb9/users/20ab0a26e89046248741b5843d323a8e/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:05.074149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5f53e-9ea2-46ed-9774-d971fc1a100a admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ebf95cc93b48648fd9db91b5cfdcb9, user_id=20ab0a26e89046248741b5843d323a8e, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:05.076870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5f53e-9ea2-46ed-9774-d971fc1a100a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:05.167597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 941/1884] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:05 2022] PUT /identity/v3/projects/87ebf95cc93b48648fd9db91b5cfdcb9/users/20ab0a26e89046248741b5843d323a8e/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:05.177840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0fbb1-c837-40fd-835b-471fbc21c76a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:05.178129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0fbb1-c837-40fd-835b-471fbc21c76a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:05.178442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0fbb1-c837-40fd-835b-471fbc21c76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:05.208010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-94e0fbb1-c837-40fd-835b-471fbc21c76a None None] MFA Rules not processed for user `20ab0a26e89046248741b5843d323a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:05.209246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-94e0fbb1-c837-40fd-835b-471fbc21c76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:05.275358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 944/1885] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:05.286360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5f53e-9ea2-46ed-9774-d971fc1a100a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:05.288176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cee5f53e-9ea2-46ed-9774-d971fc1a100a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:05.352544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5f53e-9ea2-46ed-9774-d971fc1a100a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:05.356390 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e32fa-c35c-462e-a161-2c39be006e73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:05.357059 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e32fa-c35c-462e-a161-2c39be006e73 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:05.357336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e32fa-c35c-462e-a161-2c39be006e73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:05.357608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e32fa-c35c-462e-a161-2c39be006e73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:05.366300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e32fa-c35c-462e-a161-2c39be006e73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:05.367746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e32fa-c35c-462e-a161-2c39be006e73 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:05.376909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 942/1886] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:05.693520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:05.824481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:06.390554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:06.430242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:06.544538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:06.567864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:06.730847 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:06.759963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:07.944932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:08.005395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:08.190266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:08.379338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:08.493171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d12b8-620a-4a2f-bafe-a105124495aa None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:08.493928 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d12b8-620a-4a2f-bafe-a105124495aa None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:08.494398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d12b8-620a-4a2f-bafe-a105124495aa None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:08.496795 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 945/1887] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:23:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:23:08.503414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065cd98-2b6a-4a0c-b972-38b3c8762d7a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:08.503414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065cd98-2b6a-4a0c-b972-38b3c8762d7a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:08.503414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065cd98-2b6a-4a0c-b972-38b3c8762d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:08.524291 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5065cd98-2b6a-4a0c-b972-38b3c8762d7a None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:08.526132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5065cd98-2b6a-4a0c-b972-38b3c8762d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:08.551768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:08.614044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 943/1888] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:23:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:08.631864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:08.686130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:08.755460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d12b8-620a-4a2f-bafe-a105124495aa None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:08.755460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-160d12b8-620a-4a2f-bafe-a105124495aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:08.781450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d12b8-620a-4a2f-bafe-a105124495aa None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:08.788417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee243d-d813-4c42-8e9c-a742d44edcc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:08.789464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee243d-d813-4c42-8e9c-a742d44edcc3 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:08.790382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee243d-d813-4c42-8e9c-a742d44edcc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:08.790892 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee243d-d813-4c42-8e9c-a742d44edcc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:08.800563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee243d-d813-4c42-8e9c-a742d44edcc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:08.802564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee243d-d813-4c42-8e9c-a742d44edcc3 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:08.813961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 946/1889] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:09.219669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:10.408036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065cd98-2b6a-4a0c-b972-38b3c8762d7a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:10.409461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5065cd98-2b6a-4a0c-b972-38b3c8762d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:10.463758 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:10.479981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065cd98-2b6a-4a0c-b972-38b3c8762d7a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:10.484667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:10.485316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:10.485596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:10.485867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] PATH_INFO: `/v3/users/63b2945cd5424af8b411198e66d03f1e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:10.487037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b2945cd5424af8b411198e66d03f1e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:10.488996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:10.684830 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b2945cd5424af8b411198e66d03f1e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:10.691490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b2945cd5424af8b411198e66d03f1e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:10.750133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b2945cd5424af8b411198e66d03f1e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:10.750896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b2945cd5424af8b411198e66d03f1e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:10.751723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b2945cd5424af8b411198e66d03f1e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:10.771386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 944/1890] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:10 2022] DELETE /identity/v3/users/63b2945cd5424af8b411198e66d03f1e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:10.775938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea1208-a595-41e0-b430-c56842ac7f23 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:10.776423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea1208-a595-41e0-b430-c56842ac7f23 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:10.776648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea1208-a595-41e0-b430-c56842ac7f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:10.795950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-99ea1208-a595-41e0-b430-c56842ac7f23 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:10.797147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-99ea1208-a595-41e0-b430-c56842ac7f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:10.860939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 947/1891] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:10.871283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:10.872714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:10.936153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9360a7c-c8df-45d3-9f4c-e3e0f3caf58f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:10.940733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6f775-4c87-481b-a359-297c5b013bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:10.941386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6f775-4c87-481b-a359-297c5b013bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:10.941685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6f775-4c87-481b-a359-297c5b013bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:10.941957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa6f775-4c87-481b-a359-297c5b013bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:10.952773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6f775-4c87-481b-a359-297c5b013bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:10.954260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa6f775-4c87-481b-a359-297c5b013bd2 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:10.965990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 945/1892] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:11.531695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea1208-a595-41e0-b430-c56842ac7f23 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:11.533307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-99ea1208-a595-41e0-b430-c56842ac7f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:11.550680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea1208-a595-41e0-b430-c56842ac7f23 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:11.555386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:11.556064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:11.556345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:11.556617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] PATH_INFO: `/v3/projects/b2376704712f4764a383346953b7ee63` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:11.557790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2376704712f4764a383346953b7ee63)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:11.560534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:11.686662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:11.689496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2376704712f4764a383346953b7ee63', 'request_id': 'req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:11.698647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2376704712f4764a383346953b7ee63', 'request_id': 'req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:11.704357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2376704712f4764a383346953b7ee63', 'request_id': 'req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:11.801159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] The token cache is being invalidate because project b2376704712f4764a383346953b7ee63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:11.801991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:11.802158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:11.803991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 948/1893] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:11 2022] DELETE /identity/v3/projects/b2376704712f4764a383346953b7ee63 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:11.808354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6f775-4c87-481b-a359-297c5b013bd2 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:11.818260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa6f775-4c87-481b-a359-297c5b013bd2 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:11.876031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:11.876682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:11.876967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:11.877267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] PATH_INFO: `/v3/users/a4b078f6602c484bbb3e5691d77e632a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:11.878423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b078f6602c484bbb3e5691d77e632a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:11.880448 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:11.894267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:12.042815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b078f6602c484bbb3e5691d77e632a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:12.049478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b078f6602c484bbb3e5691d77e632a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:12.113122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b078f6602c484bbb3e5691d77e632a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:12.113818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b078f6602c484bbb3e5691d77e632a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:12.114478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-dccdc9ca-7af7-488a-894c-dbc0c5774f7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b078f6602c484bbb3e5691d77e632a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:12.132237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 946/1894] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:11 2022] DELETE /identity/v3/users/a4b078f6602c484bbb3e5691d77e632a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:12.580090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:12.582035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:12.601879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2612e5d-c2d5-47d7-9b32-d20e7de917d5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:12.663356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:12.664123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:12.664411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:12.664694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] PATH_INFO: `/v3/projects/dce806ea37f2468790b5138b37aa6e5e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:12.665885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce806ea37f2468790b5138b37aa6e5e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:12.667858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:12.773106 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce806ea37f2468790b5138b37aa6e5e', 'request_id': 'req-218d79a1-4085-430d-a080-4262fa98bdb0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:12.780313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce806ea37f2468790b5138b37aa6e5e', 'request_id': 'req-218d79a1-4085-430d-a080-4262fa98bdb0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:12.786699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce806ea37f2468790b5138b37aa6e5e', 'request_id': 'req-218d79a1-4085-430d-a080-4262fa98bdb0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:12.849821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] The token cache is being invalidate because project dce806ea37f2468790b5138b37aa6e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:12.850384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:12.850722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-218d79a1-4085-430d-a080-4262fa98bdb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:12.852360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 949/1895] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:12 2022] DELETE /identity/v3/projects/dce806ea37f2468790b5138b37aa6e5e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:12.857983 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef03d4-82c5-4479-a990-d53addf4ca7a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:12.858438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef03d4-82c5-4479-a990-d53addf4ca7a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:12.858528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef03d4-82c5-4479-a990-d53addf4ca7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:12.877571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f6ef03d4-82c5-4479-a990-d53addf4ca7a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:12.878784 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f6ef03d4-82c5-4479-a990-d53addf4ca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:12.936163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 947/1896] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:12.943873 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11948164-63dc-42e1-9296-7506217e32f5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:12.944169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11948164-63dc-42e1-9296-7506217e32f5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:12.944454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11948164-63dc-42e1-9296-7506217e32f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:12.964375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-11948164-63dc-42e1-9296-7506217e32f5 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:12.966201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-11948164-63dc-42e1-9296-7506217e32f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:12.974225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 950/1897] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:12.978209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef03d4-82c5-4479-a990-d53addf4ca7a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:12.987989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef03d4-82c5-4479-a990-d53addf4ca7a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:12.992551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f5b75-312b-4840-b30a-79cfa34a46cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:12.993157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5b75-312b-4840-b30a-79cfa34a46cf admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:12.993444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5b75-312b-4840-b30a-79cfa34a46cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:12.993721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5b75-312b-4840-b30a-79cfa34a46cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:12.994412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f5b75-312b-4840-b30a-79cfa34a46cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:12.994717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f5b75-312b-4840-b30a-79cfa34a46cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:12.996339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f5b75-312b-4840-b30a-79cfa34a46cf admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:13.007695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 948/1898] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:13.011610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11948164-63dc-42e1-9296-7506217e32f5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:13.022086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11948164-63dc-42e1-9296-7506217e32f5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:13.027044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18feb722-420e-48c7-b474-8020cc896f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:13.028320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feb722-420e-48c7-b474-8020cc896f84 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:13.029509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feb722-420e-48c7-b474-8020cc896f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:13.029509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18feb722-420e-48c7-b474-8020cc896f84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:13.029994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18feb722-420e-48c7-b474-8020cc896f84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:13.031622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18feb722-420e-48c7-b474-8020cc896f84 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:13.114702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 951/1899] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:13 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:13.118671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f5b75-312b-4840-b30a-79cfa34a46cf admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:13.129348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f5b75-312b-4840-b30a-79cfa34a46cf admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:13.192356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946005fd-4d8b-4547-b32c-55531bfdfcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:13.192993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946005fd-4d8b-4547-b32c-55531bfdfcbb admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:13.193279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946005fd-4d8b-4547-b32c-55531bfdfcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:13.193586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946005fd-4d8b-4547-b32c-55531bfdfcbb admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:13.194302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946005fd-4d8b-4547-b32c-55531bfdfcbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:13.196290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946005fd-4d8b-4547-b32c-55531bfdfcbb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:13.262809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 949/1900] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:13 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:13.266997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18feb722-420e-48c7-b474-8020cc896f84 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:13.277193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18feb722-420e-48c7-b474-8020cc896f84 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:13.281808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9ac3a-9b4d-435e-a457-6f1f75628bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:13.282410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9ac3a-9b4d-435e-a457-6f1f75628bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:13.282741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9ac3a-9b4d-435e-a457-6f1f75628bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:13.283021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9ac3a-9b4d-435e-a457-6f1f75628bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:13.283706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9ac3a-9b4d-435e-a457-6f1f75628bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:13.285400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9ac3a-9b4d-435e-a457-6f1f75628bf1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:13.298280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 952/1901] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:13.311366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946005fd-4d8b-4547-b32c-55531bfdfcbb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:13.322579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946005fd-4d8b-4547-b32c-55531bfdfcbb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:13.328230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dce6ee-6b2d-4aed-b385-152e0a1b0ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:13.328838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dce6ee-6b2d-4aed-b385-152e0a1b0ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:13.329242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dce6ee-6b2d-4aed-b385-152e0a1b0ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:13.329701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dce6ee-6b2d-4aed-b385-152e0a1b0ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:13.330401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dce6ee-6b2d-4aed-b385-152e0a1b0ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:13.332058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dce6ee-6b2d-4aed-b385-152e0a1b0ca5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:13.334361 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:13.344066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 950/1902] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:13.356778 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9ac3a-9b4d-435e-a457-6f1f75628bf1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:13.367193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9ac3a-9b4d-435e-a457-6f1f75628bf1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:13.372571 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4d5af-709f-4237-8449-b02809e55829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:13.373262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4d5af-709f-4237-8449-b02809e55829 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:13.373723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4d5af-709f-4237-8449-b02809e55829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:13.374002 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4d5af-709f-4237-8449-b02809e55829 admin admin] PATH_INFO: `/v3/projects/517a8872eae54fe0a3a83848a9922bfb/users/e52cf17b6de34d0db2214e17fc2b3a96/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:13.389692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4d5af-709f-4237-8449-b02809e55829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=517a8872eae54fe0a3a83848a9922bfb, user_id=e52cf17b6de34d0db2214e17fc2b3a96, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:13.392473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4d5af-709f-4237-8449-b02809e55829 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:13.526691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:13.531832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 953/1903] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:13 2022] PUT /identity/v3/projects/517a8872eae54fe0a3a83848a9922bfb/users/e52cf17b6de34d0db2214e17fc2b3a96/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:13.543037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db542-99af-4c0f-ab50-e2e31fa8a162 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:13.543615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db542-99af-4c0f-ab50-e2e31fa8a162 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:13.543921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db542-99af-4c0f-ab50-e2e31fa8a162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:13.743817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2d1db542-99af-4c0f-ab50-e2e31fa8a162 None None] MFA Rules not processed for user `e52cf17b6de34d0db2214e17fc2b3a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:13.743817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2d1db542-99af-4c0f-ab50-e2e31fa8a162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:13.743817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 951/1904] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:13.744378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4d5af-709f-4237-8449-b02809e55829 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:13.796837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4d5af-709f-4237-8449-b02809e55829 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:13.818717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bf01c-1b34-442b-895a-0f31e1d741c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:13.820328 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bf01c-1b34-442b-895a-0f31e1d741c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:13.820785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bf01c-1b34-442b-895a-0f31e1d741c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:13.821267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bf01c-1b34-442b-895a-0f31e1d741c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:13.822416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bf01c-1b34-442b-895a-0f31e1d741c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:13.826266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bf01c-1b34-442b-895a-0f31e1d741c2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:13.921548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 954/1905] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:13 2022] POST /identity/v3/projects => generated 389 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:13.925340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1db542-99af-4c0f-ab50-e2e31fa8a162 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:13.937705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1db542-99af-4c0f-ab50-e2e31fa8a162 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:14.019018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be5e88-2e99-432c-9a9a-72f7d2ba1bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:14.019654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be5e88-2e99-432c-9a9a-72f7d2ba1bff admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:14.019968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be5e88-2e99-432c-9a9a-72f7d2ba1bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:14.020246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be5e88-2e99-432c-9a9a-72f7d2ba1bff admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:14.020923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be5e88-2e99-432c-9a9a-72f7d2ba1bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:14.023245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be5e88-2e99-432c-9a9a-72f7d2ba1bff admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:14.111254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 952/1906] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:13 2022] POST /identity/v3/users => generated 301 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:14.115952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bf01c-1b34-442b-895a-0f31e1d741c2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:14.128625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bf01c-1b34-442b-895a-0f31e1d741c2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:14.133715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779da409-ddce-47e3-9313-e1d2c5935cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:14.134380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da409-ddce-47e3-9313-e1d2c5935cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:14.134667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da409-ddce-47e3-9313-e1d2c5935cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:14.134944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da409-ddce-47e3-9313-e1d2c5935cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:14.135981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779da409-ddce-47e3-9313-e1d2c5935cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:14.138962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779da409-ddce-47e3-9313-e1d2c5935cdc admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:14.150009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 955/1907] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:14.154567 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be5e88-2e99-432c-9a9a-72f7d2ba1bff admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:14.164941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be5e88-2e99-432c-9a9a-72f7d2ba1bff admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:14.170151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f476-1369-45e1-b308-5d1df359d6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:14.172308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f476-1369-45e1-b308-5d1df359d6da admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:14.172308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f476-1369-45e1-b308-5d1df359d6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:14.172308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f476-1369-45e1-b308-5d1df359d6da admin admin] PATH_INFO: `/v3/projects/1dbd984fcde9485c8e54407737b5184f/users/c3601b945e3c4d83b5745fbb73dd1576/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:14.187794 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4f476-1369-45e1-b308-5d1df359d6da admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dbd984fcde9485c8e54407737b5184f, user_id=c3601b945e3c4d83b5745fbb73dd1576, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:14.190979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4f476-1369-45e1-b308-5d1df359d6da admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:14.316856 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 953/1908] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:14 2022] PUT /identity/v3/projects/1dbd984fcde9485c8e54407737b5184f/users/c3601b945e3c4d83b5745fbb73dd1576/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:14.321379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779da409-ddce-47e3-9313-e1d2c5935cdc admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:14.332139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779da409-ddce-47e3-9313-e1d2c5935cdc admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:14.394375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43120eed-a6db-4176-8650-318f7d5d2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:14.395052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43120eed-a6db-4176-8650-318f7d5d2bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:14.395342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43120eed-a6db-4176-8650-318f7d5d2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:14.395622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43120eed-a6db-4176-8650-318f7d5d2bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:14.396364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43120eed-a6db-4176-8650-318f7d5d2bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:14.397979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43120eed-a6db-4176-8650-318f7d5d2bf0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:14.407978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 956/1909] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:14.412308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f476-1369-45e1-b308-5d1df359d6da admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:14.428436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f476-1369-45e1-b308-5d1df359d6da admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:14.428436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cba48b-c72f-4f90-80d9-da2107f3a921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:14.429204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cba48b-c72f-4f90-80d9-da2107f3a921 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:14.429486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cba48b-c72f-4f90-80d9-da2107f3a921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:14.429779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cba48b-c72f-4f90-80d9-da2107f3a921 admin admin] PATH_INFO: `/v3/projects/1dbd984fcde9485c8e54407737b5184f/users/c3601b945e3c4d83b5745fbb73dd1576/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:14.431856 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cba48b-c72f-4f90-80d9-da2107f3a921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dbd984fcde9485c8e54407737b5184f, user_id=c3601b945e3c4d83b5745fbb73dd1576, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:14.435165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cba48b-c72f-4f90-80d9-da2107f3a921 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:14.447494 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 954/1910] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:14 2022] PUT /identity/v3/projects/1dbd984fcde9485c8e54407737b5184f/users/c3601b945e3c4d83b5745fbb73dd1576/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:14.458847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0fcad-cbd4-4a39-a1a4-ec2a50b6923a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:14.459172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0fcad-cbd4-4a39-a1a4-ec2a50b6923a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:14.459457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0fcad-cbd4-4a39-a1a4-ec2a50b6923a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:14.493607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3ee0fcad-cbd4-4a39-a1a4-ec2a50b6923a None None] MFA Rules not processed for user `c3601b945e3c4d83b5745fbb73dd1576`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:14.493607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3ee0fcad-cbd4-4a39-a1a4-ec2a50b6923a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:14.565508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:14.574744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 957/1911] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:14 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:14.687521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cba48b-c72f-4f90-80d9-da2107f3a921 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:14.689725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-13cba48b-c72f-4f90-80d9-da2107f3a921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:14.756988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:14.770721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cba48b-c72f-4f90-80d9-da2107f3a921 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:14.779243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078dcba-bb12-40bd-a090-24cedd438f65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:14.779967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078dcba-bb12-40bd-a090-24cedd438f65 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:14.780254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078dcba-bb12-40bd-a090-24cedd438f65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:14.780543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078dcba-bb12-40bd-a090-24cedd438f65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:14.789897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078dcba-bb12-40bd-a090-24cedd438f65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:14.791356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078dcba-bb12-40bd-a090-24cedd438f65 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:14.820044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 955/1912] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:14 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:15.007408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0fcad-cbd4-4a39-a1a4-ec2a50b6923a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:15.079378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0fcad-cbd4-4a39-a1a4-ec2a50b6923a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:15.084072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:15.084734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:15.085021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:15.085332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] PATH_INFO: `/v3/users/e52cf17b6de34d0db2214e17fc2b3a96` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:15.086566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e52cf17b6de34d0db2214e17fc2b3a96)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:15.088555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:15.333862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e52cf17b6de34d0db2214e17fc2b3a96'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:15.345145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e52cf17b6de34d0db2214e17fc2b3a96'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:15.441849 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e52cf17b6de34d0db2214e17fc2b3a96'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:15.442954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e52cf17b6de34d0db2214e17fc2b3a96'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:15.457438 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e52cf17b6de34d0db2214e17fc2b3a96'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:15.464032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 958/1913] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:15 2022] DELETE /identity/v3/users/e52cf17b6de34d0db2214e17fc2b3a96 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:15.469161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d10d-e7c8-4550-b2d6-d4e1bac5cbee None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:15.469731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d10d-e7c8-4550-b2d6-d4e1bac5cbee None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:15.469731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d10d-e7c8-4550-b2d6-d4e1bac5cbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:15.489567 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-cc22d10d-e7c8-4550-b2d6-d4e1bac5cbee None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:15.490769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cc22d10d-e7c8-4550-b2d6-d4e1bac5cbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:15.554831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 956/1914] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:15.566071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:15.567586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:15.632374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9ca0f-8ed5-4632-ad0d-7e911f7b55c7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:15.635996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c9b03-2d2e-48f5-a947-bf32bb69e7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:15.636759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9b03-2d2e-48f5-a947-bf32bb69e7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:15.637722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9b03-2d2e-48f5-a947-bf32bb69e7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:15.638080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c9b03-2d2e-48f5-a947-bf32bb69e7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:15.651613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c9b03-2d2e-48f5-a947-bf32bb69e7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:15.653186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c9b03-2d2e-48f5-a947-bf32bb69e7a1 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:15.666542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 959/1915] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:16.212619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22d10d-e7c8-4550-b2d6-d4e1bac5cbee None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:16.212619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cc22d10d-e7c8-4550-b2d6-d4e1bac5cbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:16.232512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22d10d-e7c8-4550-b2d6-d4e1bac5cbee None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:16.238040 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:16.238722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:16.239163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:16.239429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] PATH_INFO: `/v3/projects/517a8872eae54fe0a3a83848a9922bfb` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:16.241303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=517a8872eae54fe0a3a83848a9922bfb)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:16.247034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:16.428463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517a8872eae54fe0a3a83848a9922bfb', 'request_id': 'req-23b54b09-ecd6-4da1-96f2-71e194417878'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:16.435177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517a8872eae54fe0a3a83848a9922bfb', 'request_id': 'req-23b54b09-ecd6-4da1-96f2-71e194417878'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:16.544896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517a8872eae54fe0a3a83848a9922bfb', 'request_id': 'req-23b54b09-ecd6-4da1-96f2-71e194417878'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:16.566736 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] The token cache is being invalidate because project 517a8872eae54fe0a3a83848a9922bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:16.567992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:16.567992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:16.569959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 957/1916] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:16 2022] DELETE /identity/v3/projects/517a8872eae54fe0a3a83848a9922bfb => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:16.584635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c9b03-2d2e-48f5-a947-bf32bb69e7a1 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:16.594475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:16.596094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:16.602018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c9b03-2d2e-48f5-a947-bf32bb69e7a1 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:16.616241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:16.661163 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:16.662179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:16.662470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:16.662752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] PATH_INFO: `/v3/users/c3601b945e3c4d83b5745fbb73dd1576` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:16.663984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3601b945e3c4d83b5745fbb73dd1576)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:16.666095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:16.672243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b54b09-ecd6-4da1-96f2-71e194417878 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:16.676972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:16.677675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:16.677959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:16.678240 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] PATH_INFO: `/v3/users/7875ed3841914aeaa1f76b21b0fab66c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:16.679441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7875ed3841914aeaa1f76b21b0fab66c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:16.681414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:16.688681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:16.932982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3601b945e3c4d83b5745fbb73dd1576'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:16.943835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3601b945e3c4d83b5745fbb73dd1576'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:16.992529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7875ed3841914aeaa1f76b21b0fab66c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:16.999931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7875ed3841914aeaa1f76b21b0fab66c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.005075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3601b945e3c4d83b5745fbb73dd1576'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.006105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3601b945e3c4d83b5745fbb73dd1576'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.019503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7c088896-fda1-4183-803b-e2a94d069509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3601b945e3c4d83b5745fbb73dd1576'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.025786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 960/1917] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:16 2022] DELETE /identity/v3/users/c3601b945e3c4d83b5745fbb73dd1576 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:17.096148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7875ed3841914aeaa1f76b21b0fab66c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.097155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7875ed3841914aeaa1f76b21b0fab66c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.098236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7875ed3841914aeaa1f76b21b0fab66c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.119415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 958/1918] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:16 2022] DELETE /identity/v3/users/7875ed3841914aeaa1f76b21b0fab66c => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:17.124839 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa73863-f024-44bf-9578-fd103302a796 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:17.125139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa73863-f024-44bf-9578-fd103302a796 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:17.125420 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa73863-f024-44bf-9578-fd103302a796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:17.145479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7fa73863-f024-44bf-9578-fd103302a796 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:17.146690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7fa73863-f024-44bf-9578-fd103302a796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:17.205025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 961/1919] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:17.215305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:17.216738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:17.276562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd233f6-d20c-471c-9976-b3cd140ea851 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:17.283665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc140e22-3d5a-4632-a4a8-1892932aa223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:17.283665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc140e22-3d5a-4632-a4a8-1892932aa223 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:17.283665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc140e22-3d5a-4632-a4a8-1892932aa223 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:17.283665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc140e22-3d5a-4632-a4a8-1892932aa223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:17.290266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc140e22-3d5a-4632-a4a8-1892932aa223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:17.291701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc140e22-3d5a-4632-a4a8-1892932aa223 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:17.301354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 959/1920] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:17.472656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa73863-f024-44bf-9578-fd103302a796 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:17.474392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7fa73863-f024-44bf-9578-fd103302a796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:17.488637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa73863-f024-44bf-9578-fd103302a796 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:17.518894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:17.519563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:17.519881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:17.520159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] PATH_INFO: `/v3/projects/1dbd984fcde9485c8e54407737b5184f` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:17.521339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dbd984fcde9485c8e54407737b5184f)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:17.523240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:17.631879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:17.650335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dbd984fcde9485c8e54407737b5184f', 'request_id': 'req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.657315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dbd984fcde9485c8e54407737b5184f', 'request_id': 'req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.663770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dbd984fcde9485c8e54407737b5184f', 'request_id': 'req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.736057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] The token cache is being invalidate because project 1dbd984fcde9485c8e54407737b5184f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:17.736057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.736057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d10c50c-9a29-4831-8a79-c8dd9786d4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:17.737163 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 962/1921] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:17 2022] DELETE /identity/v3/projects/1dbd984fcde9485c8e54407737b5184f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:17.741951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923dc0a-927f-4760-a746-8760935aee33 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:17.742417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923dc0a-927f-4760-a746-8760935aee33 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:17.742517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923dc0a-927f-4760-a746-8760935aee33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:17.763144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7923dc0a-927f-4760-a746-8760935aee33 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:17.767906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7923dc0a-927f-4760-a746-8760935aee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:17.828537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 960/1922] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:17.836128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3f889-5795-48cf-b009-a966f2cb1412 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:17.836474 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3f889-5795-48cf-b009-a966f2cb1412 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:17.836826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3f889-5795-48cf-b009-a966f2cb1412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:17.857406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-51b3f889-5795-48cf-b009-a966f2cb1412 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:17.859161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-51b3f889-5795-48cf-b009-a966f2cb1412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:17.867272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 963/1923] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:17.871607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923dc0a-927f-4760-a746-8760935aee33 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:17.881397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923dc0a-927f-4760-a746-8760935aee33 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:17.886192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02556db8-48bb-4a87-95b0-a5cceeae1c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:17.886842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02556db8-48bb-4a87-95b0-a5cceeae1c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:17.887154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02556db8-48bb-4a87-95b0-a5cceeae1c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:17.887475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02556db8-48bb-4a87-95b0-a5cceeae1c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:17.888219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02556db8-48bb-4a87-95b0-a5cceeae1c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:17.888529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02556db8-48bb-4a87-95b0-a5cceeae1c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:17.890252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02556db8-48bb-4a87-95b0-a5cceeae1c1d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:17.901733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 961/1924] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:17.908111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3f889-5795-48cf-b009-a966f2cb1412 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:17.910828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02556db8-48bb-4a87-95b0-a5cceeae1c1d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:17.913163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-02556db8-48bb-4a87-95b0-a5cceeae1c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:17.918770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3f889-5795-48cf-b009-a966f2cb1412 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:17.924801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc71c81-2d91-4b68-bbb0-84e66c4205ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:17.929442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc71c81-2d91-4b68-bbb0-84e66c4205ff admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:17.929442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc71c81-2d91-4b68-bbb0-84e66c4205ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:17.929442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc71c81-2d91-4b68-bbb0-84e66c4205ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:17.929442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc71c81-2d91-4b68-bbb0-84e66c4205ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:17.929442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc71c81-2d91-4b68-bbb0-84e66c4205ff admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:17.929817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02556db8-48bb-4a87-95b0-a5cceeae1c1d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:17.934149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:17.934751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:17.935106 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:17.935313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] PATH_INFO: `/v3/projects/2a934f171be14c0dbaafe8c4dfd3d97c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:17.936659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a934f171be14c0dbaafe8c4dfd3d97c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:17.938746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:17.951555 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:18.047585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 964/1925] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:17 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:18.051620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc71c81-2d91-4b68-bbb0-84e66c4205ff admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:18.062917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a934f171be14c0dbaafe8c4dfd3d97c', 'request_id': 'req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:18.069099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a934f171be14c0dbaafe8c4dfd3d97c', 'request_id': 'req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:18.132895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc71c81-2d91-4b68-bbb0-84e66c4205ff admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:18.137813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46960aca-5e2b-4350-8f2a-bbff09a542fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:18.138434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46960aca-5e2b-4350-8f2a-bbff09a542fb admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:18.139254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46960aca-5e2b-4350-8f2a-bbff09a542fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:18.139525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46960aca-5e2b-4350-8f2a-bbff09a542fb admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:18.140551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46960aca-5e2b-4350-8f2a-bbff09a542fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:18.142136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46960aca-5e2b-4350-8f2a-bbff09a542fb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:18.149834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a934f171be14c0dbaafe8c4dfd3d97c', 'request_id': 'req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:18.166950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] The token cache is being invalidate because project 2a934f171be14c0dbaafe8c4dfd3d97c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:18.168202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:18.168202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:18.171084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 962/1926] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:17 2022] DELETE /identity/v3/projects/2a934f171be14c0dbaafe8c4dfd3d97c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:18.174856 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:18.176420 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:18.249613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdc4ef-ee3f-4f3b-94e8-d9e5b2252fd5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:18.254630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:18.255269 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:18.255549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:18.255849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] PATH_INFO: `/v3/users/935847942dbf491fafeae65d24397b18` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:18.257041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=935847942dbf491fafeae65d24397b18)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:18.257676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 965/1927] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:18 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:18.259166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:18.261699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46960aca-5e2b-4350-8f2a-bbff09a542fb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:18.263585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-46960aca-5e2b-4350-8f2a-bbff09a542fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:18.276347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46960aca-5e2b-4350-8f2a-bbff09a542fb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:18.281872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c67721-a77c-4e8c-a8c7-bb3aadfb52fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:18.282489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c67721-a77c-4e8c-a8c7-bb3aadfb52fe admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:18.286086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c67721-a77c-4e8c-a8c7-bb3aadfb52fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:18.286086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c67721-a77c-4e8c-a8c7-bb3aadfb52fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:18.286086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c67721-a77c-4e8c-a8c7-bb3aadfb52fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:18.286086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c67721-a77c-4e8c-a8c7-bb3aadfb52fe admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:18.295522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 966/1928] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:18.299336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c67721-a77c-4e8c-a8c7-bb3aadfb52fe admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:18.308994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c67721-a77c-4e8c-a8c7-bb3aadfb52fe admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:18.313554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d787d8-78b3-46bb-a6f4-16c87c244f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:18.314231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d787d8-78b3-46bb-a6f4-16c87c244f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:18.314478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d787d8-78b3-46bb-a6f4-16c87c244f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:18.314756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d787d8-78b3-46bb-a6f4-16c87c244f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:18.315437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d787d8-78b3-46bb-a6f4-16c87c244f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:18.317021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d787d8-78b3-46bb-a6f4-16c87c244f98 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:18.327919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 967/1929] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:18.331812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d787d8-78b3-46bb-a6f4-16c87c244f98 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:18.340888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d787d8-78b3-46bb-a6f4-16c87c244f98 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:18.346102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0a041-787f-4dad-a9b1-e626058d4144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:18.346727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0a041-787f-4dad-a9b1-e626058d4144 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:18.347012 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0a041-787f-4dad-a9b1-e626058d4144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:18.347286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0a041-787f-4dad-a9b1-e626058d4144 admin admin] PATH_INFO: `/v3/projects/31b00095d0744352a196be0aa9a71dec/users/0eded80eeef5465b9b3fb188298ec75f/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:18.362021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0a041-787f-4dad-a9b1-e626058d4144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b00095d0744352a196be0aa9a71dec, user_id=0eded80eeef5465b9b3fb188298ec75f, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:18.365514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0a041-787f-4dad-a9b1-e626058d4144 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:18.454491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 968/1930] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:18 2022] PUT /identity/v3/projects/31b00095d0744352a196be0aa9a71dec/users/0eded80eeef5465b9b3fb188298ec75f/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:18.464183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '935847942dbf491fafeae65d24397b18'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:18.464479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511befa0-1b55-45b8-8e71-a228f07a174c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:18.464769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511befa0-1b55-45b8-8e71-a228f07a174c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:18.465044 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511befa0-1b55-45b8-8e71-a228f07a174c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:18.473080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '935847942dbf491fafeae65d24397b18'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:18.496193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-511befa0-1b55-45b8-8e71-a228f07a174c None None] MFA Rules not processed for user `0eded80eeef5465b9b3fb188298ec75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:18.497555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-511befa0-1b55-45b8-8e71-a228f07a174c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:18.524834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:18.527153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '935847942dbf491fafeae65d24397b18'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:18.528287 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '935847942dbf491fafeae65d24397b18'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:18.529033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '935847942dbf491fafeae65d24397b18'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:18.548032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 963/1931] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:18 2022] DELETE /identity/v3/users/935847942dbf491fafeae65d24397b18 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:18.571853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 969/1932] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:18.577371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:18.578775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:18.601440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:18.691053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4bb4a-bd80-4ded-a759-917a03337209 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:18.695346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486971b7-dfb8-4dcc-83f1-b61f2fc0ed11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:18.696007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486971b7-dfb8-4dcc-83f1-b61f2fc0ed11 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:18.696288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486971b7-dfb8-4dcc-83f1-b61f2fc0ed11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:18.696561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486971b7-dfb8-4dcc-83f1-b61f2fc0ed11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:18.697235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486971b7-dfb8-4dcc-83f1-b61f2fc0ed11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:18.699162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486971b7-dfb8-4dcc-83f1-b61f2fc0ed11 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:18.811291 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 964/1933] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:18 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:18.815048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511befa0-1b55-45b8-8e71-a228f07a174c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:18.879539 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511befa0-1b55-45b8-8e71-a228f07a174c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:18.939597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10119e2-cd66-4600-87d8-2fbdea49bf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:18.940159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10119e2-cd66-4600-87d8-2fbdea49bf84 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:18.940440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10119e2-cd66-4600-87d8-2fbdea49bf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:18.940715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10119e2-cd66-4600-87d8-2fbdea49bf84 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:18.941376 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10119e2-cd66-4600-87d8-2fbdea49bf84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:18.943336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10119e2-cd66-4600-87d8-2fbdea49bf84 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.035079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486971b7-dfb8-4dcc-83f1-b61f2fc0ed11 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.036851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-486971b7-dfb8-4dcc-83f1-b61f2fc0ed11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:19.052470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486971b7-dfb8-4dcc-83f1-b61f2fc0ed11 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.063956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.063956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.063956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.063956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] PATH_INFO: `/v3/projects/b0da46341b2242719351e6c49c6b26b0` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.064638 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0da46341b2242719351e6c49c6b26b0)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.065365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.066551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 970/1934] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:18 2022] POST /identity/v3/users => generated 299 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:19.067278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10119e2-cd66-4600-87d8-2fbdea49bf84 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.077201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10119e2-cd66-4600-87d8-2fbdea49bf84 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.082497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01858a-a9e8-45d2-a657-6410f77f02a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.083110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01858a-a9e8-45d2-a657-6410f77f02a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.083393 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01858a-a9e8-45d2-a657-6410f77f02a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.083669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01858a-a9e8-45d2-a657-6410f77f02a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.084444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01858a-a9e8-45d2-a657-6410f77f02a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.089447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01858a-a9e8-45d2-a657-6410f77f02a0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.096770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 971/1935] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:19.100777 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01858a-a9e8-45d2-a657-6410f77f02a0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.111750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01858a-a9e8-45d2-a657-6410f77f02a0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.116566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502a26a-7a8c-43a8-aba1-be823dc23ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.117226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502a26a-7a8c-43a8-aba1-be823dc23ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.117509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502a26a-7a8c-43a8-aba1-be823dc23ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.117785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502a26a-7a8c-43a8-aba1-be823dc23ab0 admin admin] PATH_INFO: `/v3/projects/2bb4f324c6b748c593177ea86ed2f6b3/users/ba1459d1e286499c812ffecfc42a5d7f/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.134383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502a26a-7a8c-43a8-aba1-be823dc23ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb4f324c6b748c593177ea86ed2f6b3, user_id=ba1459d1e286499c812ffecfc42a5d7f, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.137259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502a26a-7a8c-43a8-aba1-be823dc23ab0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.202461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 972/1936] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:19 2022] PUT /identity/v3/projects/2bb4f324c6b748c593177ea86ed2f6b3/users/ba1459d1e286499c812ffecfc42a5d7f/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:19.206248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502a26a-7a8c-43a8-aba1-be823dc23ab0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.212216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0da46341b2242719351e6c49c6b26b0', 'request_id': 'req-a0fafec3-5f7d-410d-946a-72b3a728db00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:19.241904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0da46341b2242719351e6c49c6b26b0', 'request_id': 'req-a0fafec3-5f7d-410d-946a-72b3a728db00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:19.244644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:19.277955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502a26a-7a8c-43a8-aba1-be823dc23ab0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.310525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0da46341b2242719351e6c49c6b26b0', 'request_id': 'req-a0fafec3-5f7d-410d-946a-72b3a728db00'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:19.324141 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] The token cache is being invalidate because project b0da46341b2242719351e6c49c6b26b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:19.324563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:19.325258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a0fafec3-5f7d-410d-946a-72b3a728db00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:19.326979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 965/1937] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:19 2022] DELETE /identity/v3/projects/b0da46341b2242719351e6c49c6b26b0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:19.333543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd9803-7c57-4ae0-91af-ca15a5106682 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.333836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd9803-7c57-4ae0-91af-ca15a5106682 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.334183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd9803-7c57-4ae0-91af-ca15a5106682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.344504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df503fed-ff88-4ccb-9597-e17e27921f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.345404 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df503fed-ff88-4ccb-9597-e17e27921f12 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.345846 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df503fed-ff88-4ccb-9597-e17e27921f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.346266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df503fed-ff88-4ccb-9597-e17e27921f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.347259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df503fed-ff88-4ccb-9597-e17e27921f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.350285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df503fed-ff88-4ccb-9597-e17e27921f12 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.358135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7ccd9803-7c57-4ae0-91af-ca15a5106682 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:19.359357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7ccd9803-7c57-4ae0-91af-ca15a5106682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:19.363667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 973/1938] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:19.367190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df503fed-ff88-4ccb-9597-e17e27921f12 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.369194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-df503fed-ff88-4ccb-9597-e17e27921f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:19.371297 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 966/1939] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:19.379032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7b793-8d64-4c2f-9be4-fbe1708fd39d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.379322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7b793-8d64-4c2f-9be4-fbe1708fd39d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.379607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7b793-8d64-4c2f-9be4-fbe1708fd39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.383389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df503fed-ff88-4ccb-9597-e17e27921f12 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.388532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035c970-1e1a-44ec-b630-8948064d175f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.389344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c970-1e1a-44ec-b630-8948064d175f admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.389590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c970-1e1a-44ec-b630-8948064d175f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.389790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035c970-1e1a-44ec-b630-8948064d175f admin admin] PATH_INFO: `/v3/projects/2bb4f324c6b748c593177ea86ed2f6b3/users/ba1459d1e286499c812ffecfc42a5d7f/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.391961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035c970-1e1a-44ec-b630-8948064d175f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb4f324c6b748c593177ea86ed2f6b3, user_id=ba1459d1e286499c812ffecfc42a5d7f, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.403803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035c970-1e1a-44ec-b630-8948064d175f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.406066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-92c7b793-8d64-4c2f-9be4-fbe1708fd39d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:19.406444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-92c7b793-8d64-4c2f-9be4-fbe1708fd39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:19.410728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 974/1940] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:19 2022] PUT /identity/v3/projects/2bb4f324c6b748c593177ea86ed2f6b3/users/ba1459d1e286499c812ffecfc42a5d7f/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:19.422383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070afea-278a-4e8e-8fe3-6b1afce7accf None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.422684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070afea-278a-4e8e-8fe3-6b1afce7accf None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.423189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070afea-278a-4e8e-8fe3-6b1afce7accf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.452075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d070afea-278a-4e8e-8fe3-6b1afce7accf None None] MFA Rules not processed for user `ba1459d1e286499c812ffecfc42a5d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:19.453378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d070afea-278a-4e8e-8fe3-6b1afce7accf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:19.472054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 967/1941] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:19.475034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7b793-8d64-4c2f-9be4-fbe1708fd39d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.496466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7b793-8d64-4c2f-9be4-fbe1708fd39d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.501409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16611e3a-4a5b-427d-ae21-a655560c1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.502124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16611e3a-4a5b-427d-ae21-a655560c1a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.502480 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16611e3a-4a5b-427d-ae21-a655560c1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.502828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16611e3a-4a5b-427d-ae21-a655560c1a70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.503514 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16611e3a-4a5b-427d-ae21-a655560c1a70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:19.503853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16611e3a-4a5b-427d-ae21-a655560c1a70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.505502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16611e3a-4a5b-427d-ae21-a655560c1a70 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.517269 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 968/1942] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:23:19.521822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16611e3a-4a5b-427d-ae21-a655560c1a70 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.528124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 975/1943] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:19.531935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16611e3a-4a5b-427d-ae21-a655560c1a70 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.536497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f67f6-e28f-4ecd-b05f-0348c6bbdaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.537106 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f67f6-e28f-4ecd-b05f-0348c6bbdaae admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.537393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f67f6-e28f-4ecd-b05f-0348c6bbdaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.537670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f67f6-e28f-4ecd-b05f-0348c6bbdaae admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.538331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f67f6-e28f-4ecd-b05f-0348c6bbdaae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.540266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f67f6-e28f-4ecd-b05f-0348c6bbdaae admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.543894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070afea-278a-4e8e-8fe3-6b1afce7accf None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.545765 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:19.545876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d070afea-278a-4e8e-8fe3-6b1afce7accf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:19.600417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:19.618648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070afea-278a-4e8e-8fe3-6b1afce7accf None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.623412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e48993-98de-481f-8212-353f08b8d569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.624069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e48993-98de-481f-8212-353f08b8d569 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.624349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e48993-98de-481f-8212-353f08b8d569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.624622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e48993-98de-481f-8212-353f08b8d569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.634349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e48993-98de-481f-8212-353f08b8d569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.635763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e48993-98de-481f-8212-353f08b8d569 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.645623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 976/1944] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:19.687353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 969/1945] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:19 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:19.691827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e48993-98de-481f-8212-353f08b8d569 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.703498 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e48993-98de-481f-8212-353f08b8d569 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.763430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e51cb-66bb-457d-9285-1152337b7ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.764713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e51cb-66bb-457d-9285-1152337b7ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.764996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e51cb-66bb-457d-9285-1152337b7ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.765405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e51cb-66bb-457d-9285-1152337b7ec1 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.766087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e51cb-66bb-457d-9285-1152337b7ec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.768027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e51cb-66bb-457d-9285-1152337b7ec1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.882032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 977/1946] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:19 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:19.886096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f67f6-e28f-4ecd-b05f-0348c6bbdaae admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.897130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f67f6-e28f-4ecd-b05f-0348c6bbdaae admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.901674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc4eae-e204-4668-b322-945fb808b800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.902293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4eae-e204-4668-b322-945fb808b800 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.902575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4eae-e204-4668-b322-945fb808b800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.902853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc4eae-e204-4668-b322-945fb808b800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.903553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc4eae-e204-4668-b322-945fb808b800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.905772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc4eae-e204-4668-b322-945fb808b800 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.916223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 970/1947] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:19.920300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e51cb-66bb-457d-9285-1152337b7ec1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.930515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e51cb-66bb-457d-9285-1152337b7ec1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.935471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9689fc-bf17-4ef1-a357-32ce32038867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.936181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9689fc-bf17-4ef1-a357-32ce32038867 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.936451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9689fc-bf17-4ef1-a357-32ce32038867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.936871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9689fc-bf17-4ef1-a357-32ce32038867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.937484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9689fc-bf17-4ef1-a357-32ce32038867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.939125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9689fc-bf17-4ef1-a357-32ce32038867 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:19.953671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 978/1948] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:19.954633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc4eae-e204-4668-b322-945fb808b800 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:19.965713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc4eae-e204-4668-b322-945fb808b800 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:19.970435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f92fa-8309-477f-a5d2-bf32e7ad4dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:19.971117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f92fa-8309-477f-a5d2-bf32e7ad4dff admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:19.976237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f92fa-8309-477f-a5d2-bf32e7ad4dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:19.976237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f92fa-8309-477f-a5d2-bf32e7ad4dff admin admin] PATH_INFO: `/v3/projects/b3e07a1bd3fe4a9e93cce4072171106b/users/b886e2ae832d4c0f9e54aedf964e17d5/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:19.987981 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f92fa-8309-477f-a5d2-bf32e7ad4dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e07a1bd3fe4a9e93cce4072171106b, user_id=b886e2ae832d4c0f9e54aedf964e17d5, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:19.990844 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f92fa-8309-477f-a5d2-bf32e7ad4dff admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:20.070523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 971/1949] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:19 2022] PUT /identity/v3/projects/b3e07a1bd3fe4a9e93cce4072171106b/users/b886e2ae832d4c0f9e54aedf964e17d5/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:20.081762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2e37f-f594-419a-b42c-ab503604bf32 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:20.082057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2e37f-f594-419a-b42c-ab503604bf32 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:20.082341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2e37f-f594-419a-b42c-ab503604bf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:20.112201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d7b2e37f-f594-419a-b42c-ab503604bf32 None None] MFA Rules not processed for user `b886e2ae832d4c0f9e54aedf964e17d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:20.113430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d7b2e37f-f594-419a-b42c-ab503604bf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:20.179987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 979/1950] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:20.184660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f92fa-8309-477f-a5d2-bf32e7ad4dff admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:20.252050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f92fa-8309-477f-a5d2-bf32e7ad4dff admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:20.257284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfb262-df53-4082-accc-6f683a247118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:20.257916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfb262-df53-4082-accc-6f683a247118 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:20.258200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfb262-df53-4082-accc-6f683a247118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:20.258471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfb262-df53-4082-accc-6f683a247118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:20.259131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfb262-df53-4082-accc-6f683a247118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:20.261087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfb262-df53-4082-accc-6f683a247118 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:20.367710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 972/1951] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:20 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:20.368709 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2e37f-f594-419a-b42c-ab503604bf32 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:20.385364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2e37f-f594-419a-b42c-ab503604bf32 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:20.465390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:20.493225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe8cae-2d29-4ebf-aee1-d1cdf6908ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:20.494288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe8cae-2d29-4ebf-aee1-d1cdf6908ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:20.494976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe8cae-2d29-4ebf-aee1-d1cdf6908ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:20.496034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe8cae-2d29-4ebf-aee1-d1cdf6908ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:20.497573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe8cae-2d29-4ebf-aee1-d1cdf6908ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:20.500744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe8cae-2d29-4ebf-aee1-d1cdf6908ed7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:20.640448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 980/1952] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:20 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:20.641872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfb262-df53-4082-accc-6f683a247118 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:20.653360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfb262-df53-4082-accc-6f683a247118 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:20.663680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc31b-39f7-456a-9016-cb6ba138b298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:20.664515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc31b-39f7-456a-9016-cb6ba138b298 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:20.664809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc31b-39f7-456a-9016-cb6ba138b298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:20.665096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc31b-39f7-456a-9016-cb6ba138b298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:20.665834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc31b-39f7-456a-9016-cb6ba138b298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:20.674457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc31b-39f7-456a-9016-cb6ba138b298 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:20.691841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 973/1953] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:20.695896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe8cae-2d29-4ebf-aee1-d1cdf6908ed7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:20.709448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe8cae-2d29-4ebf-aee1-d1cdf6908ed7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:20.715623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243334bf-68f0-4916-87ed-1583cda3a148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:20.716432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243334bf-68f0-4916-87ed-1583cda3a148 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:20.717476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243334bf-68f0-4916-87ed-1583cda3a148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:20.718007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243334bf-68f0-4916-87ed-1583cda3a148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:20.718904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243334bf-68f0-4916-87ed-1583cda3a148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:20.720660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243334bf-68f0-4916-87ed-1583cda3a148 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:20.736387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 981/1954] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:20.740768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc31b-39f7-456a-9016-cb6ba138b298 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:20.754374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc31b-39f7-456a-9016-cb6ba138b298 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:20.773197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b36806-447b-4f22-ac51-dd5d7ab492d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:20.774010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b36806-447b-4f22-ac51-dd5d7ab492d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:20.774392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b36806-447b-4f22-ac51-dd5d7ab492d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:20.774744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b36806-447b-4f22-ac51-dd5d7ab492d1 admin admin] PATH_INFO: `/v3/projects/fa8677a2552c445988c0c2f29a952455/users/f36dae8fc79e4898bd1576559c9e0a0a/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:20.790143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b36806-447b-4f22-ac51-dd5d7ab492d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa8677a2552c445988c0c2f29a952455, user_id=f36dae8fc79e4898bd1576559c9e0a0a, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:20.793799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b36806-447b-4f22-ac51-dd5d7ab492d1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:20.887395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 974/1955] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:20 2022] PUT /identity/v3/projects/fa8677a2552c445988c0c2f29a952455/users/f36dae8fc79e4898bd1576559c9e0a0a/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:20.898823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec6961-d772-419c-b304-cf34819bafc9 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:20.899282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec6961-d772-419c-b304-cf34819bafc9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:20.899400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec6961-d772-419c-b304-cf34819bafc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:20.918880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b36806-447b-4f22-ac51-dd5d7ab492d1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:20.936303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-08ec6961-d772-419c-b304-cf34819bafc9 None None] MFA Rules not processed for user `f36dae8fc79e4898bd1576559c9e0a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:20.938412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-08ec6961-d772-419c-b304-cf34819bafc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:21.019464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b36806-447b-4f22-ac51-dd5d7ab492d1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:21.025544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:21.027542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:21.028038 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:21.028489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] PATH_INFO: `/v3/users/0eded80eeef5465b9b3fb188298ec75f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:21.032095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eded80eeef5465b9b3fb188298ec75f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:21.035572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 982/1956] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:21.036413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:21.037855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec6961-d772-419c-b304-cf34819bafc9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:21.041726 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:21.050752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec6961-d772-419c-b304-cf34819bafc9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:21.055708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0191a6-ceb5-404e-90ba-d4d23d241d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:21.056444 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0191a6-ceb5-404e-90ba-d4d23d241d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:21.056782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0191a6-ceb5-404e-90ba-d4d23d241d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:21.057071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0191a6-ceb5-404e-90ba-d4d23d241d93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:21.057788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0191a6-ceb5-404e-90ba-d4d23d241d93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:21.059702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0191a6-ceb5-404e-90ba-d4d23d241d93 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:21.217123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 983/1957] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:21 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:21.217123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0191a6-ceb5-404e-90ba-d4d23d241d93 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:21.253921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eded80eeef5465b9b3fb188298ec75f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:21.262970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eded80eeef5465b9b3fb188298ec75f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:21.329052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0191a6-ceb5-404e-90ba-d4d23d241d93 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:21.339058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd7282-ffca-4380-ac58-9602d82a1cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:21.339707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd7282-ffca-4380-ac58-9602d82a1cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:21.340035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd7282-ffca-4380-ac58-9602d82a1cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:21.340315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd7282-ffca-4380-ac58-9602d82a1cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:21.341054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd7282-ffca-4380-ac58-9602d82a1cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:21.343144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd7282-ffca-4380-ac58-9602d82a1cf7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:21.356874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eded80eeef5465b9b3fb188298ec75f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:21.358647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eded80eeef5465b9b3fb188298ec75f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:21.359566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f58ccf86-b9da-4d6c-a862-45fd19714c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eded80eeef5465b9b3fb188298ec75f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:21.416458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 975/1958] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:20 2022] DELETE /identity/v3/users/0eded80eeef5465b9b3fb188298ec75f => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:21.423215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f23dee-a8a0-4958-a3de-a7d1c1df5b57 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:21.424473 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f23dee-a8a0-4958-a3de-a7d1c1df5b57 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:21.425161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f23dee-a8a0-4958-a3de-a7d1c1df5b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:21.448778 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 984/1959] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:21 2022] POST /identity/v3/users => generated 296 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:21.454614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd7282-ffca-4380-ac58-9602d82a1cf7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:21.460886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-73cd7282-ffca-4380-ac58-9602d82a1cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:21.464498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-42f23dee-a8a0-4958-a3de-a7d1c1df5b57 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:21.467333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-42f23dee-a8a0-4958-a3de-a7d1c1df5b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:21.510273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd7282-ffca-4380-ac58-9602d82a1cf7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:21.567581 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:21.605003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 976/1960] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:21.606353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f23dee-a8a0-4958-a3de-a7d1c1df5b57 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:21.608774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-42f23dee-a8a0-4958-a3de-a7d1c1df5b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:21.624446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f23dee-a8a0-4958-a3de-a7d1c1df5b57 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:21.630396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996f709-2060-4c0b-a041-d4df6fcba664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:21.631768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996f709-2060-4c0b-a041-d4df6fcba664 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:21.632022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:21.632730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996f709-2060-4c0b-a041-d4df6fcba664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:21.632910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:21.633951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:21.633951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] PATH_INFO: `/v3/users/20ab0a26e89046248741b5843d323a8e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:21.634628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ab0a26e89046248741b5843d323a8e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:21.635129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996f709-2060-4c0b-a041-d4df6fcba664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:21.636982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:21.638378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996f709-2060-4c0b-a041-d4df6fcba664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:21.648718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996f709-2060-4c0b-a041-d4df6fcba664 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:21.661509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 985/1961] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:21.661949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996f709-2060-4c0b-a041-d4df6fcba664 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:21.666821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3996f709-2060-4c0b-a041-d4df6fcba664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:21.727348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996f709-2060-4c0b-a041-d4df6fcba664 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:21.731429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81462d56-428b-4e65-b333-ad3a7bf65a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:21.732167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81462d56-428b-4e65-b333-ad3a7bf65a59 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:21.732610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81462d56-428b-4e65-b333-ad3a7bf65a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:21.733481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81462d56-428b-4e65-b333-ad3a7bf65a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:21.743881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81462d56-428b-4e65-b333-ad3a7bf65a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:21.745627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81462d56-428b-4e65-b333-ad3a7bf65a59 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:21.760417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 986/1962] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:21.762583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81462d56-428b-4e65-b333-ad3a7bf65a59 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:21.772193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81462d56-428b-4e65-b333-ad3a7bf65a59 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:21.777713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e6ad8-b357-4cd2-b58b-35133d6bcc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:21.778559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e6ad8-b357-4cd2-b58b-35133d6bcc6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:21.778843 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e6ad8-b357-4cd2-b58b-35133d6bcc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:21.779125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e6ad8-b357-4cd2-b58b-35133d6bcc6a admin admin] PATH_INFO: `/v3/projects/5f71edbe0d564667a9828ba045eb23e8/users/47c781420ca546c199914cf13efccdbe/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:21.822079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e6ad8-b357-4cd2-b58b-35133d6bcc6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f71edbe0d564667a9828ba045eb23e8, user_id=47c781420ca546c199914cf13efccdbe, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:21.824591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e6ad8-b357-4cd2-b58b-35133d6bcc6a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:21.876019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ab0a26e89046248741b5843d323a8e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:21.889667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ab0a26e89046248741b5843d323a8e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:21.945393 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 987/1963] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:21 2022] PUT /identity/v3/projects/5f71edbe0d564667a9828ba045eb23e8/users/47c781420ca546c199914cf13efccdbe/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:21.949106 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e6ad8-b357-4cd2-b58b-35133d6bcc6a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:22.005044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ab0a26e89046248741b5843d323a8e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:22.005817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ab0a26e89046248741b5843d323a8e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:22.006429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7121cda2-4794-4910-89e8-a752b7242d76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ab0a26e89046248741b5843d323a8e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:22.015594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e6ad8-b357-4cd2-b58b-35133d6bcc6a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:22.021642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288c0c7-0eb8-4606-8f48-9b065a48e4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:22.022538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288c0c7-0eb8-4606-8f48-9b065a48e4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:22.022538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288c0c7-0eb8-4606-8f48-9b065a48e4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:22.022834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288c0c7-0eb8-4606-8f48-9b065a48e4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:22.023923 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288c0c7-0eb8-4606-8f48-9b065a48e4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:22.024233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:22.025568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288c0c7-0eb8-4606-8f48-9b065a48e4e8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:22.036893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 988/1964] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:22.038835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 977/1965] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:21 2022] DELETE /identity/v3/users/20ab0a26e89046248741b5843d323a8e => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:22.041457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288c0c7-0eb8-4606-8f48-9b065a48e4e8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:22.043992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b288c0c7-0eb8-4606-8f48-9b065a48e4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:22.044126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a572ad2-a365-4f91-b652-f5a0dd641d2d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:22.044126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a572ad2-a365-4f91-b652-f5a0dd641d2d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:22.044126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a572ad2-a365-4f91-b652-f5a0dd641d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:22.059364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288c0c7-0eb8-4606-8f48-9b065a48e4e8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:22.073482 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8a572ad2-a365-4f91-b652-f5a0dd641d2d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:22.073482 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8a572ad2-a365-4f91-b652-f5a0dd641d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:22.135385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:22.160415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e44a89-905a-4871-92bd-d40335355fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:22.161604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44a89-905a-4871-92bd-d40335355fab admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:22.162003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44a89-905a-4871-92bd-d40335355fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:22.162414 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44a89-905a-4871-92bd-d40335355fab admin admin] PATH_INFO: `/v3/projects/5f71edbe0d564667a9828ba045eb23e8/users/47c781420ca546c199914cf13efccdbe/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:22.166895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e44a89-905a-4871-92bd-d40335355fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f71edbe0d564667a9828ba045eb23e8, user_id=47c781420ca546c199914cf13efccdbe, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:22.171549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e44a89-905a-4871-92bd-d40335355fab admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:22.187258 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 989/1966] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:22 2022] PUT /identity/v3/projects/5f71edbe0d564667a9828ba045eb23e8/users/47c781420ca546c199914cf13efccdbe/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:22.198245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851c81a-9c5d-4f1f-b990-ce9590cd8b76 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:22.198677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851c81a-9c5d-4f1f-b990-ce9590cd8b76 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:22.198975 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851c81a-9c5d-4f1f-b990-ce9590cd8b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:22.203996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 978/1967] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:22.221891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a572ad2-a365-4f91-b652-f5a0dd641d2d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:22.223732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8a572ad2-a365-4f91-b652-f5a0dd641d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:22.237491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6851c81a-9c5d-4f1f-b990-ce9590cd8b76 None None] MFA Rules not processed for user `47c781420ca546c199914cf13efccdbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:22.238844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6851c81a-9c5d-4f1f-b990-ce9590cd8b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:22.321419 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a572ad2-a365-4f91-b652-f5a0dd641d2d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:22.401322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 990/1968] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:22.401621 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851c81a-9c5d-4f1f-b990-ce9590cd8b76 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:22.403193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6851c81a-9c5d-4f1f-b990-ce9590cd8b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:22.475354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c578a-0157-4aa9-8a7b-3ee417700249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:22.476509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c578a-0157-4aa9-8a7b-3ee417700249 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:22.477075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c578a-0157-4aa9-8a7b-3ee417700249 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:22.477359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c578a-0157-4aa9-8a7b-3ee417700249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:22.519850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851c81a-9c5d-4f1f-b990-ce9590cd8b76 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:22.524787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:22.525467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:22.525750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:22.526191 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] PATH_INFO: `/v3/projects/31b00095d0744352a196be0aa9a71dec` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:22.527807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b00095d0744352a196be0aa9a71dec)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:22.530840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:22.637627 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c578a-0157-4aa9-8a7b-3ee417700249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:22.639767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c578a-0157-4aa9-8a7b-3ee417700249 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:22.644332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:22.663236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b00095d0744352a196be0aa9a71dec', 'request_id': 'req-878de0ee-b33a-4380-8b44-f739fd53e55f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:22.663483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 979/1969] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:22.667600 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c578a-0157-4aa9-8a7b-3ee417700249 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:22.669147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-677c578a-0157-4aa9-8a7b-3ee417700249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:22.671492 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b00095d0744352a196be0aa9a71dec', 'request_id': 'req-878de0ee-b33a-4380-8b44-f739fd53e55f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:22.678032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:22.680026 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b00095d0744352a196be0aa9a71dec', 'request_id': 'req-878de0ee-b33a-4380-8b44-f739fd53e55f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:22.766219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c578a-0157-4aa9-8a7b-3ee417700249 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:22.783569 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] The token cache is being invalidate because project 31b00095d0744352a196be0aa9a71dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:22.784055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:22.785055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:22.787268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 991/1970] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:22 2022] DELETE /identity/v3/projects/31b00095d0744352a196be0aa9a71dec => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:22.791351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:22.792930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:22.846685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e8370-2fda-4c45-9b64-00a0f568e80c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:22.847284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e8370-2fda-4c45-9b64-00a0f568e80c service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:22.847863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e8370-2fda-4c45-9b64-00a0f568e80c service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:22.848292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e8370-2fda-4c45-9b64-00a0f568e80c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:22.850198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f55e8370-2fda-4c45-9b64-00a0f568e80c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:22.874682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878de0ee-b33a-4380-8b44-f739fd53e55f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:22.882559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:22.884949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:22.884949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:22.884949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] PATH_INFO: `/v3/users/ba1459d1e286499c812ffecfc42a5d7f` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:22.885672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba1459d1e286499c812ffecfc42a5d7f)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:22.887716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:22.945887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e8370-2fda-4c45-9b64-00a0f568e80c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:22.947650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e8370-2fda-4c45-9b64-00a0f568e80c service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:22.960001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 980/1971] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:23.104062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e8370-2fda-4c45-9b64-00a0f568e80c service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:23.105088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f55e8370-2fda-4c45-9b64-00a0f568e80c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:23.117400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e8370-2fda-4c45-9b64-00a0f568e80c service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:23.119474 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba1459d1e286499c812ffecfc42a5d7f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.125656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e4998-8ef3-49bf-87cd-50445bf8e49b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:23.126356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e4998-8ef3-49bf-87cd-50445bf8e49b service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:23.127029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e4998-8ef3-49bf-87cd-50445bf8e49b service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:23.127029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e4998-8ef3-49bf-87cd-50445bf8e49b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:23.128959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8f4e4998-8ef3-49bf-87cd-50445bf8e49b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:23.130896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba1459d1e286499c812ffecfc42a5d7f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.186292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba1459d1e286499c812ffecfc42a5d7f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.186998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba1459d1e286499c812ffecfc42a5d7f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.196989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e4998-8ef3-49bf-87cd-50445bf8e49b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:23.198595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e4998-8ef3-49bf-87cd-50445bf8e49b service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:23.204726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba1459d1e286499c812ffecfc42a5d7f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.209997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 981/1972] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:23.213177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 992/1973] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:22 2022] DELETE /identity/v3/users/ba1459d1e286499c812ffecfc42a5d7f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:23.464984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e4998-8ef3-49bf-87cd-50445bf8e49b service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:23.468541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8f4e4998-8ef3-49bf-87cd-50445bf8e49b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:23.481099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:23.487499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e4998-8ef3-49bf-87cd-50445bf8e49b service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:23.605080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:23.605821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:23.606702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:23.606702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] PATH_INFO: `/v3/projects/87ebf95cc93b48648fd9db91b5cfdcb9` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:23.608411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ebf95cc93b48648fd9db91b5cfdcb9)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:23.610905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:23.707754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ebf95cc93b48648fd9db91b5cfdcb9', 'request_id': 'req-c56175ec-719b-4ad3-97b7-7c4fb6107148'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.714001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ebf95cc93b48648fd9db91b5cfdcb9', 'request_id': 'req-c56175ec-719b-4ad3-97b7-7c4fb6107148'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.729045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:23.731502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:23.757523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1018e3f-b2f2-479b-8b07-b1631e92eb0b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:23.812074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ebf95cc93b48648fd9db91b5cfdcb9', 'request_id': 'req-c56175ec-719b-4ad3-97b7-7c4fb6107148'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.821296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:23.822241 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:23.825970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:23.825970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] PATH_INFO: `/v3/projects/2bb4f324c6b748c593177ea86ed2f6b3` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:23.825970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb4f324c6b748c593177ea86ed2f6b3)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:23.827003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] The token cache is being invalidate because project 87ebf95cc93b48648fd9db91b5cfdcb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:23.827003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.827141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:23.827332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c56175ec-719b-4ad3-97b7-7c4fb6107148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.829366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 982/1974] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:23 2022] DELETE /identity/v3/projects/87ebf95cc93b48648fd9db91b5cfdcb9 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:23.834522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cec29d-18c2-4e40-8e6d-65a39ea89ab3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:23.834880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cec29d-18c2-4e40-8e6d-65a39ea89ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:23.835246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cec29d-18c2-4e40-8e6d-65a39ea89ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:23.838780 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:23.858236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-52cec29d-18c2-4e40-8e6d-65a39ea89ab3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:23.859386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-52cec29d-18c2-4e40-8e6d-65a39ea89ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:23.867764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 983/1975] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:23.880210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf523cc-1ef5-42ab-82eb-6a135aa27b48 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:23.880700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf523cc-1ef5-42ab-82eb-6a135aa27b48 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:23.881032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf523cc-1ef5-42ab-82eb-6a135aa27b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:23.900837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-faf523cc-1ef5-42ab-82eb-6a135aa27b48 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:23.906552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-faf523cc-1ef5-42ab-82eb-6a135aa27b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:23.910774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 984/1976] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:23.914953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf523cc-1ef5-42ab-82eb-6a135aa27b48 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:23.926197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf523cc-1ef5-42ab-82eb-6a135aa27b48 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:23.931656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf55d0b-09a5-4b0b-aff2-a40861ce2301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:23.932329 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf55d0b-09a5-4b0b-aff2-a40861ce2301 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:23.932614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf55d0b-09a5-4b0b-aff2-a40861ce2301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:23.936271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf55d0b-09a5-4b0b-aff2-a40861ce2301 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:23.936271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf55d0b-09a5-4b0b-aff2-a40861ce2301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:23.936271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf55d0b-09a5-4b0b-aff2-a40861ce2301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:23.936271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf55d0b-09a5-4b0b-aff2-a40861ce2301 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:23.948179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 985/1977] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:23.953219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf55d0b-09a5-4b0b-aff2-a40861ce2301 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:23.963155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf55d0b-09a5-4b0b-aff2-a40861ce2301 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:23.974363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb4f324c6b748c593177ea86ed2f6b3', 'request_id': 'req-90e4770a-c442-4b44-abd8-933450a7ba1e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.981560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb4f324c6b748c593177ea86ed2f6b3', 'request_id': 'req-90e4770a-c442-4b44-abd8-933450a7ba1e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:23.987395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb4f324c6b748c593177ea86ed2f6b3', 'request_id': 'req-90e4770a-c442-4b44-abd8-933450a7ba1e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:24.029069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9a3a8-4819-4338-851f-67a9856e37e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.029888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9a3a8-4819-4338-851f-67a9856e37e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.030186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9a3a8-4819-4338-851f-67a9856e37e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.030458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9a3a8-4819-4338-851f-67a9856e37e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.031148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9a3a8-4819-4338-851f-67a9856e37e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.033188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9a3a8-4819-4338-851f-67a9856e37e9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.045058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:24.092015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] The token cache is being invalidate because project 2bb4f324c6b748c593177ea86ed2f6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:24.092489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:24.093275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-90e4770a-c442-4b44-abd8-933450a7ba1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:24.098465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 993/1978] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:23 2022] DELETE /identity/v3/projects/2bb4f324c6b748c593177ea86ed2f6b3 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:24.100579 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec2b78-fcc2-4e01-bf02-8ac8ceae69cd None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.101051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec2b78-fcc2-4e01-bf02-8ac8ceae69cd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.101515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec2b78-fcc2-4e01-bf02-8ac8ceae69cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.128101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ffec2b78-fcc2-4e01-bf02-8ac8ceae69cd None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:24.129326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ffec2b78-fcc2-4e01-bf02-8ac8ceae69cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:24.145669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 986/1979] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:23 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:24.146040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 994/1980] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:24.147621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71538899-d696-4800-9fb5-a82635b9691f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.148248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71538899-d696-4800-9fb5-a82635b9691f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.149076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71538899-d696-4800-9fb5-a82635b9691f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.150156 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec2b78-fcc2-4e01-bf02-8ac8ceae69cd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.151887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ffec2b78-fcc2-4e01-bf02-8ac8ceae69cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:24.178132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec2b78-fcc2-4e01-bf02-8ac8ceae69cd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.184789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-71538899-d696-4800-9fb5-a82635b9691f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:24.186664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-71538899-d696-4800-9fb5-a82635b9691f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:24.282598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ae55f-1690-4a04-ac25-5a191e6afbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.283601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae55f-1690-4a04-ac25-5a191e6afbc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.284250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae55f-1690-4a04-ac25-5a191e6afbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.284756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae55f-1690-4a04-ac25-5a191e6afbc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.285637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ae55f-1690-4a04-ac25-5a191e6afbc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.287906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ae55f-1690-4a04-ac25-5a191e6afbc1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.288037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 987/1981] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:24.292723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71538899-d696-4800-9fb5-a82635b9691f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.304250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71538899-d696-4800-9fb5-a82635b9691f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.309049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67393b89-4f92-403d-b4f1-04ddcc6d2328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.309671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393b89-4f92-403d-b4f1-04ddcc6d2328 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.309953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393b89-4f92-403d-b4f1-04ddcc6d2328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.310285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393b89-4f92-403d-b4f1-04ddcc6d2328 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.311720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67393b89-4f92-403d-b4f1-04ddcc6d2328 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:24.312051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67393b89-4f92-403d-b4f1-04ddcc6d2328 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.313651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67393b89-4f92-403d-b4f1-04ddcc6d2328 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.326367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 988/1982] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:23:24.330585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67393b89-4f92-403d-b4f1-04ddcc6d2328 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.341243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67393b89-4f92-403d-b4f1-04ddcc6d2328 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.348519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fb931-3435-47ae-91a9-90952b0894ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.349353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fb931-3435-47ae-91a9-90952b0894ef admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.349483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fb931-3435-47ae-91a9-90952b0894ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.349940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fb931-3435-47ae-91a9-90952b0894ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.350619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fb931-3435-47ae-91a9-90952b0894ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.353811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fb931-3435-47ae-91a9-90952b0894ef admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.399417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 995/1983] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:24 2022] POST /identity/v3/users => generated 288 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:24.400659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ae55f-1690-4a04-ac25-5a191e6afbc1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.417267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ae55f-1690-4a04-ac25-5a191e6afbc1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.431310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5701dd0-b8d8-4aae-a680-b58fbe31d643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.432410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5701dd0-b8d8-4aae-a680-b58fbe31d643 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.432877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5701dd0-b8d8-4aae-a680-b58fbe31d643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.433694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5701dd0-b8d8-4aae-a680-b58fbe31d643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.434821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5701dd0-b8d8-4aae-a680-b58fbe31d643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.441028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5701dd0-b8d8-4aae-a680-b58fbe31d643 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.461372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 996/1984] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:24.465486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5701dd0-b8d8-4aae-a680-b58fbe31d643 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.476663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5701dd0-b8d8-4aae-a680-b58fbe31d643 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.483092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3db5b5-2f14-4ae7-bde1-f72513f7fe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.483623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3db5b5-2f14-4ae7-bde1-f72513f7fe0e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.483956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3db5b5-2f14-4ae7-bde1-f72513f7fe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.484293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3db5b5-2f14-4ae7-bde1-f72513f7fe0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.485071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3db5b5-2f14-4ae7-bde1-f72513f7fe0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.486962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3db5b5-2f14-4ae7-bde1-f72513f7fe0e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.492341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 989/1985] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:24 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:24.497437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fb931-3435-47ae-91a9-90952b0894ef admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.504620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 997/1986] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:24.505693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3db5b5-2f14-4ae7-bde1-f72513f7fe0e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.516895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3db5b5-2f14-4ae7-bde1-f72513f7fe0e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.590487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fb931-3435-47ae-91a9-90952b0894ef admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.605915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2138835-5130-4334-88c7-287b089f637e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.607458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2138835-5130-4334-88c7-287b089f637e admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.608668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2138835-5130-4334-88c7-287b089f637e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.609306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2138835-5130-4334-88c7-287b089f637e admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.611338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2138835-5130-4334-88c7-287b089f637e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.613234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2138835-5130-4334-88c7-287b089f637e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.618772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a350e-61a0-4304-8d9b-926e845f6c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.622596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a350e-61a0-4304-8d9b-926e845f6c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.622596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a350e-61a0-4304-8d9b-926e845f6c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.622596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a350e-61a0-4304-8d9b-926e845f6c73 admin admin] PATH_INFO: `/v3/projects/12d945a3be564132969417814adecd60/users/32c44b9df53349dfae48b8e42ade37ab/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.651016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a350e-61a0-4304-8d9b-926e845f6c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12d945a3be564132969417814adecd60, user_id=32c44b9df53349dfae48b8e42ade37ab, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.654119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a350e-61a0-4304-8d9b-926e845f6c73 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.734285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 990/1987] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:24 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:24.736812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2138835-5130-4334-88c7-287b089f637e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.756385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2138835-5130-4334-88c7-287b089f637e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.761266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfb9fe-5c5c-45a9-a6ea-22afac999316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.761897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfb9fe-5c5c-45a9-a6ea-22afac999316 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.762182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfb9fe-5c5c-45a9-a6ea-22afac999316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.762456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfb9fe-5c5c-45a9-a6ea-22afac999316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.763153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfb9fe-5c5c-45a9-a6ea-22afac999316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.765265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfb9fe-5c5c-45a9-a6ea-22afac999316 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.781218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 991/1988] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:24.782228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfb9fe-5c5c-45a9-a6ea-22afac999316 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.784361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 998/1989] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:24 2022] PUT /identity/v3/projects/12d945a3be564132969417814adecd60/users/32c44b9df53349dfae48b8e42ade37ab/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:24.793770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfb9fe-5c5c-45a9-a6ea-22afac999316 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.796899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6abfb-bdcd-46ce-8397-9df88abd75d8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.797515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6abfb-bdcd-46ce-8397-9df88abd75d8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.798034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6abfb-bdcd-46ce-8397-9df88abd75d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.837509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3af6abfb-bdcd-46ce-8397-9df88abd75d8 None None] MFA Rules not processed for user `32c44b9df53349dfae48b8e42ade37ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:24.838845 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3af6abfb-bdcd-46ce-8397-9df88abd75d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:24.859222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbd624-47e8-4db1-8e78-0111cefd1f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.860006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbd624-47e8-4db1-8e78-0111cefd1f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.860291 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbd624-47e8-4db1-8e78-0111cefd1f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.860568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbd624-47e8-4db1-8e78-0111cefd1f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.861264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbd624-47e8-4db1-8e78-0111cefd1f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.862967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbd624-47e8-4db1-8e78-0111cefd1f6e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.873470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 992/1990] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:24.877600 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbd624-47e8-4db1-8e78-0111cefd1f6e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.888138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbd624-47e8-4db1-8e78-0111cefd1f6e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.893550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1f8b9-a32e-49c3-93fc-6be647a804ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.895388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1f8b9-a32e-49c3-93fc-6be647a804ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.895388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1f8b9-a32e-49c3-93fc-6be647a804ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.895388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1f8b9-a32e-49c3-93fc-6be647a804ed admin admin] PATH_INFO: `/v3/projects/b74f6d8bd4ee404a8481f4b384472cdd/users/bb2250d3bac949a394266a915fa0cec5/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.908644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 999/1991] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:24.914060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6abfb-bdcd-46ce-8397-9df88abd75d8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:24.915551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1f8b9-a32e-49c3-93fc-6be647a804ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b74f6d8bd4ee404a8481f4b384472cdd, user_id=bb2250d3bac949a394266a915fa0cec5, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.918467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1f8b9-a32e-49c3-93fc-6be647a804ed admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.924984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6abfb-bdcd-46ce-8397-9df88abd75d8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:24.929680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae550f-7519-441c-92b9-daf096ca4e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:24.930308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae550f-7519-441c-92b9-daf096ca4e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:24.930590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae550f-7519-441c-92b9-daf096ca4e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:24.930866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae550f-7519-441c-92b9-daf096ca4e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:24.931542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae550f-7519-441c-92b9-daf096ca4e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:24.933460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae550f-7519-441c-92b9-daf096ca4e80 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:24.999855 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 993/1992] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:24 2022] PUT /identity/v3/projects/b74f6d8bd4ee404a8481f4b384472cdd/users/bb2250d3bac949a394266a915fa0cec5/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:25.011086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe293ac-e516-4228-847e-394694ac29fc None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.011369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe293ac-e516-4228-847e-394694ac29fc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.011649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe293ac-e516-4228-847e-394694ac29fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.042250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1000/1993] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:24 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:25.042606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bfe293ac-e516-4228-847e-394694ac29fc None None] MFA Rules not processed for user `bb2250d3bac949a394266a915fa0cec5`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:25.044147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bfe293ac-e516-4228-847e-394694ac29fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:25.045633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae550f-7519-441c-92b9-daf096ca4e80 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:25.122148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:25.134521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 994/1994] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:25.136026 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae550f-7519-441c-92b9-daf096ca4e80 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:25.142910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe23dd-76b3-4db5-8bc0-aa8254f1c4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:25.143540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe23dd-76b3-4db5-8bc0-aa8254f1c4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.143932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe23dd-76b3-4db5-8bc0-aa8254f1c4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.144243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe23dd-76b3-4db5-8bc0-aa8254f1c4cb admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.144916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe23dd-76b3-4db5-8bc0-aa8254f1c4cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:25.146954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe23dd-76b3-4db5-8bc0-aa8254f1c4cb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:25.147904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe293ac-e516-4228-847e-394694ac29fc None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:25.149820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bfe293ac-e516-4228-847e-394694ac29fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:25.221211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe293ac-e516-4228-847e-394694ac29fc None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:25.227185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d25eb0-b072-47bb-a7ba-f17a280e8b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:25.228054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d25eb0-b072-47bb-a7ba-f17a280e8b99 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.228370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d25eb0-b072-47bb-a7ba-f17a280e8b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.228610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d25eb0-b072-47bb-a7ba-f17a280e8b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.242967 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d25eb0-b072-47bb-a7ba-f17a280e8b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:25.244803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d25eb0-b072-47bb-a7ba-f17a280e8b99 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:25.260625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1001/1995] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:25 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:25.261032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 995/1996] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:25.265486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe23dd-76b3-4db5-8bc0-aa8254f1c4cb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:25.282274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe23dd-76b3-4db5-8bc0-aa8254f1c4cb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:25.287331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:25.289008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d25eb0-b072-47bb-a7ba-f17a280e8b99 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:25.290880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d594d-ad9f-4787-8381-bc6cca545ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:25.301057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d594d-ad9f-4787-8381-bc6cca545ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.301057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d594d-ad9f-4787-8381-bc6cca545ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.301057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d594d-ad9f-4787-8381-bc6cca545ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.301057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d594d-ad9f-4787-8381-bc6cca545ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:25.301057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d594d-ad9f-4787-8381-bc6cca545ca4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:25.301526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d25eb0-b072-47bb-a7ba-f17a280e8b99 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:25.313489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:25.314367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.314829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.315279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] PATH_INFO: `/v3/users/bb2250d3bac949a394266a915fa0cec5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.316938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb2250d3bac949a394266a915fa0cec5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:25.317426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1002/1997] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:25.319278 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:25.332284 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d594d-ad9f-4787-8381-bc6cca545ca4 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:25.348477 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d594d-ad9f-4787-8381-bc6cca545ca4 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:25.360493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345daab-4a15-441c-8e8e-8a24119832f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:25.361500 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345daab-4a15-441c-8e8e-8a24119832f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.361714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345daab-4a15-441c-8e8e-8a24119832f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.362101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345daab-4a15-441c-8e8e-8a24119832f7 admin admin] PATH_INFO: `/v3/projects/e01602271f974f97b67562f35983a29a/users/5417047080c545bfa31dbbcd9caa617b/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.378999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345daab-4a15-441c-8e8e-8a24119832f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01602271f974f97b67562f35983a29a, user_id=5417047080c545bfa31dbbcd9caa617b, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:25.385535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345daab-4a15-441c-8e8e-8a24119832f7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:25.501998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1003/1998] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:25 2022] PUT /identity/v3/projects/e01602271f974f97b67562f35983a29a/users/5417047080c545bfa31dbbcd9caa617b/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:25.506274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345daab-4a15-441c-8e8e-8a24119832f7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:25.583003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb2250d3bac949a394266a915fa0cec5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:25.596207 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb2250d3bac949a394266a915fa0cec5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:25.617369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345daab-4a15-441c-8e8e-8a24119832f7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:25.622780 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fba37-6e0e-4da9-84ef-3783b4669f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:25.624031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fba37-6e0e-4da9-84ef-3783b4669f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.624318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fba37-6e0e-4da9-84ef-3783b4669f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.624597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fba37-6e0e-4da9-84ef-3783b4669f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.625295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fba37-6e0e-4da9-84ef-3783b4669f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:25.626893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fba37-6e0e-4da9-84ef-3783b4669f02 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:25.651918 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1004/1999] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:25.653784 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fba37-6e0e-4da9-84ef-3783b4669f02 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:25.657367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb2250d3bac949a394266a915fa0cec5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:25.658490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb2250d3bac949a394266a915fa0cec5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:25.658702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5770c72b-6f60-4114-b152-47c2554922e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb2250d3bac949a394266a915fa0cec5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:25.676907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fba37-6e0e-4da9-84ef-3783b4669f02 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:25.683829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ac65e-ff8e-4d61-8707-ef80ce2e59fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:25.685293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac65e-ff8e-4d61-8707-ef80ce2e59fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.686302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac65e-ff8e-4d61-8707-ef80ce2e59fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.686679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac65e-ff8e-4d61-8707-ef80ce2e59fe admin admin] PATH_INFO: `/v3/projects/e01602271f974f97b67562f35983a29a/users/5417047080c545bfa31dbbcd9caa617b/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.690299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ac65e-ff8e-4d61-8707-ef80ce2e59fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01602271f974f97b67562f35983a29a, user_id=5417047080c545bfa31dbbcd9caa617b, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:25.690618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 996/2000] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:25 2022] DELETE /identity/v3/users/bb2250d3bac949a394266a915fa0cec5 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:25.692425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb1802-d03e-4664-b4ec-7296dee15f58 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.692867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb1802-d03e-4664-b4ec-7296dee15f58 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.693292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb1802-d03e-4664-b4ec-7296dee15f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.693692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ac65e-ff8e-4d61-8707-ef80ce2e59fe admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:25.707362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1005/2001] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:25 2022] PUT /identity/v3/projects/e01602271f974f97b67562f35983a29a/users/5417047080c545bfa31dbbcd9caa617b/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:25.718545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-6adb1802-d03e-4664-b4ec-7296dee15f58 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:25.720305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6adb1802-d03e-4664-b4ec-7296dee15f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:25.726473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511165a-e7c4-4809-bd99-dfd55e615111 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.726473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511165a-e7c4-4809-bd99-dfd55e615111 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.726473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511165a-e7c4-4809-bd99-dfd55e615111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.756781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:25.808340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9511165a-e7c4-4809-bd99-dfd55e615111 None None] MFA Rules not processed for user `5417047080c545bfa31dbbcd9caa617b`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:25.809565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9511165a-e7c4-4809-bd99-dfd55e615111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:25.825529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:25.835637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 997/2002] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:25.847951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb1802-d03e-4664-b4ec-7296dee15f58 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:25.850276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6adb1802-d03e-4664-b4ec-7296dee15f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:25.937221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1006/2003] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:25.947980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb1802-d03e-4664-b4ec-7296dee15f58 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:25.952090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e2c0e-c7ee-4b93-8161-484e2949b1cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:25.952719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e2c0e-c7ee-4b93-8161-484e2949b1cc service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:25.952994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e2c0e-c7ee-4b93-8161-484e2949b1cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:25.953264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e2c0e-c7ee-4b93-8161-484e2949b1cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:25.964984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511165a-e7c4-4809-bd99-dfd55e615111 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:25.966401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e2c0e-c7ee-4b93-8161-484e2949b1cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:25.966576 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9511165a-e7c4-4809-bd99-dfd55e615111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:25.968744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e2c0e-c7ee-4b93-8161-484e2949b1cc service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:25.979439 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 998/2004] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:26.036389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511165a-e7c4-4809-bd99-dfd55e615111 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:26.040387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242c145-56d0-48ab-9c21-9d7789a88220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:26.042505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242c145-56d0-48ab-9c21-9d7789a88220 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:26.042505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242c145-56d0-48ab-9c21-9d7789a88220 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:26.042505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242c145-56d0-48ab-9c21-9d7789a88220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:26.043900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0242c145-56d0-48ab-9c21-9d7789a88220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:26.104129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242c145-56d0-48ab-9c21-9d7789a88220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:26.105866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242c145-56d0-48ab-9c21-9d7789a88220 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:26.115684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1007/2005] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:26.211509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:26.419421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e2c0e-c7ee-4b93-8161-484e2949b1cc service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:26.419421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-445e2c0e-c7ee-4b93-8161-484e2949b1cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:26.427188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e2c0e-c7ee-4b93-8161-484e2949b1cc service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:26.432859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:26.433393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:26.433610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:26.433890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] PATH_INFO: `/v3/projects/b74f6d8bd4ee404a8481f4b384472cdd` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:26.435162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b74f6d8bd4ee404a8481f4b384472cdd)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:26.437345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:26.463164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242c145-56d0-48ab-9c21-9d7789a88220 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:26.466669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0242c145-56d0-48ab-9c21-9d7789a88220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:26.482836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242c145-56d0-48ab-9c21-9d7789a88220 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:26.487845 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:26.488504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:26.488776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:26.489051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] PATH_INFO: `/v3/users/762519187671476ba4738b2fbd9ec6f8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:26.490208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=762519187671476ba4738b2fbd9ec6f8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:26.492159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:26.591123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b74f6d8bd4ee404a8481f4b384472cdd', 'request_id': 'req-360cedf7-32bb-4472-aea7-cf4f62bcb395'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:26.593169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:26.597943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b74f6d8bd4ee404a8481f4b384472cdd', 'request_id': 'req-360cedf7-32bb-4472-aea7-cf4f62bcb395'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:26.693392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b74f6d8bd4ee404a8481f4b384472cdd', 'request_id': 'req-360cedf7-32bb-4472-aea7-cf4f62bcb395'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:26.705710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] The token cache is being invalidate because project b74f6d8bd4ee404a8481f4b384472cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:26.706001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:26.706734 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-360cedf7-32bb-4472-aea7-cf4f62bcb395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:26.708475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 999/2006] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:26 2022] DELETE /identity/v3/projects/b74f6d8bd4ee404a8481f4b384472cdd => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:26.713178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7bfba-6442-4fcf-bfc7-48c49873ba0d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:26.713509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7bfba-6442-4fcf-bfc7-48c49873ba0d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:26.713850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7bfba-6442-4fcf-bfc7-48c49873ba0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:26.734780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-83f7bfba-6442-4fcf-bfc7-48c49873ba0d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:26.736011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-83f7bfba-6442-4fcf-bfc7-48c49873ba0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:26.756924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762519187671476ba4738b2fbd9ec6f8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:26.777724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762519187671476ba4738b2fbd9ec6f8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:26.801854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1000/2007] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:26.809386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f33a2-41da-4a19-9cc1-ac9286479116 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:26.809695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f33a2-41da-4a19-9cc1-ac9286479116 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:26.809952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f33a2-41da-4a19-9cc1-ac9286479116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:26.831145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f41f33a2-41da-4a19-9cc1-ac9286479116 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:26.832373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f41f33a2-41da-4a19-9cc1-ac9286479116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:26.840175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1001/2008] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:26.844622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f33a2-41da-4a19-9cc1-ac9286479116 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:26.845221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762519187671476ba4738b2fbd9ec6f8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:26.846233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762519187671476ba4738b2fbd9ec6f8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:26.853484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f33a2-41da-4a19-9cc1-ac9286479116 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:26.858777 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2a504-19cd-4276-a377-b08fa851db57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:26.859088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2a504-19cd-4276-a377-b08fa851db57 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:26.859372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2a504-19cd-4276-a377-b08fa851db57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:26.859641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2a504-19cd-4276-a377-b08fa851db57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:26.860366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2a504-19cd-4276-a377-b08fa851db57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:26.860669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2a504-19cd-4276-a377-b08fa851db57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:26.860822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d3c1099e-54f0-457c-bc80-55cad2317331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762519187671476ba4738b2fbd9ec6f8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:26.862279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2a504-19cd-4276-a377-b08fa851db57 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:26.867343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1008/2009] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:26 2022] DELETE /identity/v3/users/762519187671476ba4738b2fbd9ec6f8 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:26.872285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fca41-1cc2-4f7e-8827-dd7f350751a7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:26.872708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fca41-1cc2-4f7e-8827-dd7f350751a7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:26.877355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fca41-1cc2-4f7e-8827-dd7f350751a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:26.880234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1002/2010] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:23:26.881113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2a504-19cd-4276-a377-b08fa851db57 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:26.882790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2ac2a504-19cd-4276-a377-b08fa851db57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:26.896153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2a504-19cd-4276-a377-b08fa851db57 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:26.899737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-809fca41-1cc2-4f7e-8827-dd7f350751a7 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:26.901122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-809fca41-1cc2-4f7e-8827-dd7f350751a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:26.955753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3bb2d-e4bc-4309-81af-9b0ff6c0fadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:26.956410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3bb2d-e4bc-4309-81af-9b0ff6c0fadd admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:26.956691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3bb2d-e4bc-4309-81af-9b0ff6c0fadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:26.956975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3bb2d-e4bc-4309-81af-9b0ff6c0fadd admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:26.957660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3bb2d-e4bc-4309-81af-9b0ff6c0fadd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:26.959618 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3bb2d-e4bc-4309-81af-9b0ff6c0fadd admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:26.964599 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1009/2011] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:26.974300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fca41-1cc2-4f7e-8827-dd7f350751a7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:26.975682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-809fca41-1cc2-4f7e-8827-dd7f350751a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:27.024006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:27.037978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fca41-1cc2-4f7e-8827-dd7f350751a7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.041694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0c89a-b717-418f-aba9-771733d905a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.042361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0c89a-b717-418f-aba9-771733d905a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.042592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0c89a-b717-418f-aba9-771733d905a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.042877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0c89a-b717-418f-aba9-771733d905a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.052468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1003/2012] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:26 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:27.053556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3bb2d-e4bc-4309-81af-9b0ff6c0fadd admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.055659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0c89a-b717-418f-aba9-771733d905a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.057215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0c89a-b717-418f-aba9-771733d905a5 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.069114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1010/2013] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:27.119769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3bb2d-e4bc-4309-81af-9b0ff6c0fadd admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.125177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a256f2-8045-4022-a541-1c848426297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.125810 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a256f2-8045-4022-a541-1c848426297d admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.126096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a256f2-8045-4022-a541-1c848426297d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.126376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a256f2-8045-4022-a541-1c848426297d admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.127051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a256f2-8045-4022-a541-1c848426297d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.129116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a256f2-8045-4022-a541-1c848426297d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.211178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1004/2014] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:27 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:27.215311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0c89a-b717-418f-aba9-771733d905a5 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.224990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0c89a-b717-418f-aba9-771733d905a5 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.230173 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1e3b9-d28d-40c1-8eea-a0bf9c30ac53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.230782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e3b9-d28d-40c1-8eea-a0bf9c30ac53 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.231067 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e3b9-d28d-40c1-8eea-a0bf9c30ac53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.231344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e3b9-d28d-40c1-8eea-a0bf9c30ac53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.232292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1e3b9-d28d-40c1-8eea-a0bf9c30ac53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.234434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1e3b9-d28d-40c1-8eea-a0bf9c30ac53 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.244288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1011/2015] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:27.248794 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a256f2-8045-4022-a541-1c848426297d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.255266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:27.260528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a256f2-8045-4022-a541-1c848426297d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.265147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026ed3e-8df7-4aa1-afcc-571c7c717483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.265767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ed3e-8df7-4aa1-afcc-571c7c717483 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.266052 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ed3e-8df7-4aa1-afcc-571c7c717483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.266336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ed3e-8df7-4aa1-afcc-571c7c717483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.267024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ed3e-8df7-4aa1-afcc-571c7c717483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.268685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ed3e-8df7-4aa1-afcc-571c7c717483 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.278648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1005/2016] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:27.283077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1e3b9-d28d-40c1-8eea-a0bf9c30ac53 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.293401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1e3b9-d28d-40c1-8eea-a0bf9c30ac53 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.298677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2800a-31d2-4a48-a336-0d42c8f5729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.299374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2800a-31d2-4a48-a336-0d42c8f5729e admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.299660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2800a-31d2-4a48-a336-0d42c8f5729e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.299971 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2800a-31d2-4a48-a336-0d42c8f5729e admin admin] PATH_INFO: `/v3/projects/2935fa5f518c4d5c8a05a5677770d1cc/users/640ba0bfca094f3bbc130c2cddd6f2c0/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.315753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2800a-31d2-4a48-a336-0d42c8f5729e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2935fa5f518c4d5c8a05a5677770d1cc, user_id=640ba0bfca094f3bbc130c2cddd6f2c0, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.318531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2800a-31d2-4a48-a336-0d42c8f5729e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.383069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1012/2017] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:27 2022] PUT /identity/v3/projects/2935fa5f518c4d5c8a05a5677770d1cc/users/640ba0bfca094f3bbc130c2cddd6f2c0/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:27.399540 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dc2f7-40e6-4dd2-9f0e-8d58861f3094 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.399949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dc2f7-40e6-4dd2-9f0e-8d58861f3094 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.400299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dc2f7-40e6-4dd2-9f0e-8d58861f3094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.430463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-805dc2f7-40e6-4dd2-9f0e-8d58861f3094 None None] MFA Rules not processed for user `640ba0bfca094f3bbc130c2cddd6f2c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:27.431658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-805dc2f7-40e6-4dd2-9f0e-8d58861f3094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:27.503135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1006/2018] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:27.514565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2800a-31d2-4a48-a336-0d42c8f5729e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.516409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9ae2800a-31d2-4a48-a336-0d42c8f5729e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:27.540513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dc2f7-40e6-4dd2-9f0e-8d58861f3094 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.542281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-805dc2f7-40e6-4dd2-9f0e-8d58861f3094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:27.583628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2800a-31d2-4a48-a336-0d42c8f5729e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.588024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91287450-5165-4dc6-9c3e-687284829ccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.589293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91287450-5165-4dc6-9c3e-687284829ccf service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.589293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91287450-5165-4dc6-9c3e-687284829ccf service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.589512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91287450-5165-4dc6-9c3e-687284829ccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.599562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91287450-5165-4dc6-9c3e-687284829ccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.601218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91287450-5165-4dc6-9c3e-687284829ccf service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.611596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1013/2019] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:27.618273 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dc2f7-40e6-4dd2-9f0e-8d58861f3094 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.623224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.623911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.624197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.624476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] PATH_INFO: `/v3/projects/d05c0fac722645dc9eee9074d3354781` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.625644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05c0fac722645dc9eee9074d3354781)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.627568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.778140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05c0fac722645dc9eee9074d3354781', 'request_id': 'req-258a3eb2-2cc9-4010-9471-8432643dc2f8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:27.784966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05c0fac722645dc9eee9074d3354781', 'request_id': 'req-258a3eb2-2cc9-4010-9471-8432643dc2f8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:27.846956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05c0fac722645dc9eee9074d3354781', 'request_id': 'req-258a3eb2-2cc9-4010-9471-8432643dc2f8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:27.858864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] The token cache is being invalidate because project d05c0fac722645dc9eee9074d3354781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:27.859154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:27.859806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:27.861349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1007/2020] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:27 2022] DELETE /identity/v3/projects/d05c0fac722645dc9eee9074d3354781 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:27.865288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91287450-5165-4dc6-9c3e-687284829ccf service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.875629 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91287450-5165-4dc6-9c3e-687284829ccf service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.939960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.939960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.939960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.939960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] PATH_INFO: `/v3/users/34277a0b4681446bb3920420edd2b5d8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.941109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=34277a0b4681446bb3920420edd2b5d8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.943058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:27.964454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:27.968657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:27.981077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a3eb2-2cc9-4010-9471-8432643dc2f8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:27.985974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:27.986563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:27.986849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:27.987116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] PATH_INFO: `/v3/users/b886e2ae832d4c0f9e54aedf964e17d5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:27.988314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b886e2ae832d4c0f9e54aedf964e17d5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:27.990312 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:28.175536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34277a0b4681446bb3920420edd2b5d8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.179933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:28.184484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34277a0b4681446bb3920420edd2b5d8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.221144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b886e2ae832d4c0f9e54aedf964e17d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.228033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b886e2ae832d4c0f9e54aedf964e17d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.284017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34277a0b4681446bb3920420edd2b5d8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.284729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34277a0b4681446bb3920420edd2b5d8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.299981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f9eb9621-71bd-4e52-8def-b52a57379d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34277a0b4681446bb3920420edd2b5d8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.316096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:28.323329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1014/2021] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:27 2022] DELETE /identity/v3/users/34277a0b4681446bb3920420edd2b5d8 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:28.344973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b886e2ae832d4c0f9e54aedf964e17d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.345611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b886e2ae832d4c0f9e54aedf964e17d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.346377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b886e2ae832d4c0f9e54aedf964e17d5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.387393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1008/2022] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:27 2022] DELETE /identity/v3/users/b886e2ae832d4c0f9e54aedf964e17d5 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:28.392620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654efe1-2c57-4d2b-9d1c-31a272813464 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:28.392898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654efe1-2c57-4d2b-9d1c-31a272813464 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:28.397059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654efe1-2c57-4d2b-9d1c-31a272813464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:28.413107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7654efe1-2c57-4d2b-9d1c-31a272813464 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:28.414332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7654efe1-2c57-4d2b-9d1c-31a272813464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:28.477279 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1015/2023] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:28.487675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:28.489300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:28.558403 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f302620-b09c-4932-8523-3138de127b3f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:28.562584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4e480-7376-49dc-9b29-c562a300609d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:28.563294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4e480-7376-49dc-9b29-c562a300609d service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:28.563649 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4e480-7376-49dc-9b29-c562a300609d service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:28.564025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4e480-7376-49dc-9b29-c562a300609d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:28.574650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4e480-7376-49dc-9b29-c562a300609d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:28.580360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4e480-7376-49dc-9b29-c562a300609d service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:28.588239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1009/2024] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:28.821762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654efe1-2c57-4d2b-9d1c-31a272813464 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:28.823696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7654efe1-2c57-4d2b-9d1c-31a272813464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:28.840438 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654efe1-2c57-4d2b-9d1c-31a272813464 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:28.845255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:28.846091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:28.846374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:28.846652 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] PATH_INFO: `/v3/projects/d066c13020cc45b6a43ce4cb60461d50` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:28.847841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d066c13020cc45b6a43ce4cb60461d50)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:28.849761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:28.986319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd066c13020cc45b6a43ce4cb60461d50', 'request_id': 'req-86a3d36d-35af-43de-b0ce-727438037377'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:28.994936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd066c13020cc45b6a43ce4cb60461d50', 'request_id': 'req-86a3d36d-35af-43de-b0ce-727438037377'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.001141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd066c13020cc45b6a43ce4cb60461d50', 'request_id': 'req-86a3d36d-35af-43de-b0ce-727438037377'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.049476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4e480-7376-49dc-9b29-c562a300609d service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:29.050914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6bb4e480-7376-49dc-9b29-c562a300609d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:29.065628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4e480-7376-49dc-9b29-c562a300609d service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:29.079245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] The token cache is being invalidate because project d066c13020cc45b6a43ce4cb60461d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:29.079722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.080391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-86a3d36d-35af-43de-b0ce-727438037377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.082347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1016/2025] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:28 2022] DELETE /identity/v3/projects/d066c13020cc45b6a43ce4cb60461d50 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:29.086818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fde0c-45bf-4fbc-a718-49444c7a1ef7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.087112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fde0c-45bf-4fbc-a718-49444c7a1ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.087398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fde0c-45bf-4fbc-a718-49444c7a1ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.106662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6b0fde0c-45bf-4fbc-a718-49444c7a1ef7 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:29.107910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6b0fde0c-45bf-4fbc-a718-49444c7a1ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:29.124078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:29.125480 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.125903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.126257 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] PATH_INFO: `/v3/projects/b3e07a1bd3fe4a9e93cce4072171106b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.127488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e07a1bd3fe4a9e93cce4072171106b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:29.129752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:29.167600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1017/2026] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:29.175932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfffffd-f838-48a7-a618-dfb9a3ab32f5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.176253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfffffd-f838-48a7-a618-dfb9a3ab32f5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.176534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfffffd-f838-48a7-a618-dfb9a3ab32f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.199948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-cdfffffd-f838-48a7-a618-dfb9a3ab32f5 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:29.201176 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-cdfffffd-f838-48a7-a618-dfb9a3ab32f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:29.206249 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:29.211319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1018/2027] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:29.216504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfffffd-f838-48a7-a618-dfb9a3ab32f5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:29.225818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfffffd-f838-48a7-a618-dfb9a3ab32f5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:29.227949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e07a1bd3fe4a9e93cce4072171106b', 'request_id': 'req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.238793 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e07a1bd3fe4a9e93cce4072171106b', 'request_id': 'req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.275683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:29.283337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1f6cc-e8e9-4049-b3c9-588dfa9f8c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:29.284002 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1f6cc-e8e9-4049-b3c9-588dfa9f8c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.284290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1f6cc-e8e9-4049-b3c9-588dfa9f8c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.284908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1f6cc-e8e9-4049-b3c9-588dfa9f8c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.285329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1f6cc-e8e9-4049-b3c9-588dfa9f8c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:29.285684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1f6cc-e8e9-4049-b3c9-588dfa9f8c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:29.287348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1f6cc-e8e9-4049-b3c9-588dfa9f8c1d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:29.301198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1019/2028] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:29.303970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1f6cc-e8e9-4049-b3c9-588dfa9f8c1d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:29.313980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1f6cc-e8e9-4049-b3c9-588dfa9f8c1d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:29.318821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a539c1-e9be-44fd-bee6-b47890139d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:29.319055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e07a1bd3fe4a9e93cce4072171106b', 'request_id': 'req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.319421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a539c1-e9be-44fd-bee6-b47890139d11 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.319705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a539c1-e9be-44fd-bee6-b47890139d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.320017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a539c1-e9be-44fd-bee6-b47890139d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.320684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a539c1-e9be-44fd-bee6-b47890139d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:29.322554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a539c1-e9be-44fd-bee6-b47890139d11 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:29.331284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] The token cache is being invalidate because project b3e07a1bd3fe4a9e93cce4072171106b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:29.331704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.332395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.333942 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1010/2029] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:29 2022] DELETE /identity/v3/projects/b3e07a1bd3fe4a9e93cce4072171106b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:29.337784 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:29.339180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:29.352666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea7210-0cca-4ae1-92a0-fa5d5708be13 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:29.357775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:29.358428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.360050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.360395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] PATH_INFO: `/v3/users/f36dae8fc79e4898bd1576559c9e0a0a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.361613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f36dae8fc79e4898bd1576559c9e0a0a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:29.363611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:29.417333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1020/2030] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:29 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:29.418584 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a539c1-e9be-44fd-bee6-b47890139d11 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:29.420578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c2a539c1-e9be-44fd-bee6-b47890139d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:29.509306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a539c1-e9be-44fd-bee6-b47890139d11 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:29.513989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:29.514635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.520127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.520127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] PATH_INFO: `/v3/users/32c44b9df53349dfae48b8e42ade37ab` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.520127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c44b9df53349dfae48b8e42ade37ab)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:29.520127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:29.600154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f36dae8fc79e4898bd1576559c9e0a0a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.608041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f36dae8fc79e4898bd1576559c9e0a0a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.664192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f36dae8fc79e4898bd1576559c9e0a0a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.664827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f36dae8fc79e4898bd1576559c9e0a0a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.665441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f36dae8fc79e4898bd1576559c9e0a0a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.678387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c44b9df53349dfae48b8e42ade37ab'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.684368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1011/2031] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:29 2022] DELETE /identity/v3/users/f36dae8fc79e4898bd1576559c9e0a0a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:29.687701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:29.687885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c44b9df53349dfae48b8e42ade37ab'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.687985 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:29.761071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df0f1d-6efd-4e8e-8350-2dae87d62381 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:29.766889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b3320-61c5-4f46-b975-a2fab3dd4fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:29.767527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b3320-61c5-4f46-b975-a2fab3dd4fcb admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.767839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b3320-61c5-4f46-b975-a2fab3dd4fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.768118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b3320-61c5-4f46-b975-a2fab3dd4fcb admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.768787 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b3320-61c5-4f46-b975-a2fab3dd4fcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:29.770771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b3320-61c5-4f46-b975-a2fab3dd4fcb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:29.782713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c44b9df53349dfae48b8e42ade37ab'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.783531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c44b9df53349dfae48b8e42ade37ab'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.795990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5cab3094-a0c5-40ff-b370-0266199715e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c44b9df53349dfae48b8e42ade37ab'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:29.802026 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1021/2032] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:29 2022] DELETE /identity/v3/users/32c44b9df53349dfae48b8e42ade37ab => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:29.806364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae0901-f432-40d7-9766-d3524d9846d8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.806909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae0901-f432-40d7-9766-d3524d9846d8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.807373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae0901-f432-40d7-9766-d3524d9846d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.836231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f3ae0901-f432-40d7-9766-d3524d9846d8 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:29.841544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f3ae0901-f432-40d7-9766-d3524d9846d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:29.904425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1022/2033] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:29.915471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae0901-f432-40d7-9766-d3524d9846d8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:29.917007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f3ae0901-f432-40d7-9766-d3524d9846d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:29.926013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1012/2034] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:29 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:29.927702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b3320-61c5-4f46-b975-a2fab3dd4fcb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:29.929669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7b4b3320-61c5-4f46-b975-a2fab3dd4fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:29.943884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b3320-61c5-4f46-b975-a2fab3dd4fcb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:29.949839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05b187-dc4d-4654-ab43-99c341f35498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:29.950774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05b187-dc4d-4654-ab43-99c341f35498 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.951251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05b187-dc4d-4654-ab43-99c341f35498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.951576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05b187-dc4d-4654-ab43-99c341f35498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.952455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05b187-dc4d-4654-ab43-99c341f35498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:29.955077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05b187-dc4d-4654-ab43-99c341f35498 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:29.976710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1013/2035] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:29.985437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05b187-dc4d-4654-ab43-99c341f35498 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:29.986100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae0901-f432-40d7-9766-d3524d9846d8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:29.989976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a4ead-dc37-441d-b9a7-6ddd2a1ce8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:29.990708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a4ead-dc37-441d-b9a7-6ddd2a1ce8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:29.991099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a4ead-dc37-441d-b9a7-6ddd2a1ce8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:29.992441 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a4ead-dc37-441d-b9a7-6ddd2a1ce8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:29.995310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05b187-dc4d-4654-ab43-99c341f35498 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.001119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886c605-636f-4846-b9d6-d3aa5f5ea42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.001738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886c605-636f-4846-b9d6-d3aa5f5ea42e admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.002017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886c605-636f-4846-b9d6-d3aa5f5ea42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.002234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a4ead-dc37-441d-b9a7-6ddd2a1ce8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.002335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886c605-636f-4846-b9d6-d3aa5f5ea42e admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.002971 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886c605-636f-4846-b9d6-d3aa5f5ea42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.004450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a4ead-dc37-441d-b9a7-6ddd2a1ce8d3 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.004608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886c605-636f-4846-b9d6-d3aa5f5ea42e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.016053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1014/2036] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:30.016533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1023/2037] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:30.020288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886c605-636f-4846-b9d6-d3aa5f5ea42e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:30.032345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886c605-636f-4846-b9d6-d3aa5f5ea42e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.037381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d32600-92e0-40ea-aa24-d675879f8b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.038131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d32600-92e0-40ea-aa24-d675879f8b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.038422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d32600-92e0-40ea-aa24-d675879f8b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.038704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d32600-92e0-40ea-aa24-d675879f8b0f admin admin] PATH_INFO: `/v3/projects/b6efdb494bf9472ebbd10ba0bbc9c13c/users/81880947f409400c93e1694fab6e4d67/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.054760 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d32600-92e0-40ea-aa24-d675879f8b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6efdb494bf9472ebbd10ba0bbc9c13c, user_id=81880947f409400c93e1694fab6e4d67, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.057723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d32600-92e0-40ea-aa24-d675879f8b0f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.127467 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:30.201389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1015/2038] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:30 2022] PUT /identity/v3/projects/b6efdb494bf9472ebbd10ba0bbc9c13c/users/81880947f409400c93e1694fab6e4d67/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:30.212403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92c404-db70-4f89-b70e-60354cdb1d81 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.212697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92c404-db70-4f89-b70e-60354cdb1d81 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.212981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92c404-db70-4f89-b70e-60354cdb1d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.219314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:30.227212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d32600-92e0-40ea-aa24-d675879f8b0f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:30.229918 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-42d32600-92e0-40ea-aa24-d675879f8b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:30.264596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1f92c404-db70-4f89-b70e-60354cdb1d81 None None] MFA Rules not processed for user `81880947f409400c93e1694fab6e4d67`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:30.269078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1f92c404-db70-4f89-b70e-60354cdb1d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:30.322903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d32600-92e0-40ea-aa24-d675879f8b0f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.331727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.332461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.332752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.333015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] PATH_INFO: `/v3/projects/fa8677a2552c445988c0c2f29a952455` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.339025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa8677a2552c445988c0c2f29a952455)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.339025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.355392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1024/2039] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:30.367745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92c404-db70-4f89-b70e-60354cdb1d81 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:30.367745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1f92c404-db70-4f89-b70e-60354cdb1d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:30.377280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92c404-db70-4f89-b70e-60354cdb1d81 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.383419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.384104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.384440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.384743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] PATH_INFO: `/v3/users/da550a1e87a148baa154dd5e0b03b11d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.385978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] RBAC: Authorizing `identity:delete_user(user_id=da550a1e87a148baa154dd5e0b03b11d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.388032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.447142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa8677a2552c445988c0c2f29a952455', 'request_id': 'req-0a276244-e7ad-435c-8436-740d2a14054f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:30.452382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa8677a2552c445988c0c2f29a952455', 'request_id': 'req-0a276244-e7ad-435c-8436-740d2a14054f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:30.510156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa8677a2552c445988c0c2f29a952455', 'request_id': 'req-0a276244-e7ad-435c-8436-740d2a14054f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:30.522724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] The token cache is being invalidate because project fa8677a2552c445988c0c2f29a952455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:30.523016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:30.523724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:30.525384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1016/2040] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:30 2022] DELETE /identity/v3/projects/fa8677a2552c445988c0c2f29a952455 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:30.528315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:30.528315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:30.592940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a276244-e7ad-435c-8436-740d2a14054f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.598461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7991106-0dd5-4544-8a92-318af40327ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.599342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7991106-0dd5-4544-8a92-318af40327ae admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.599696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7991106-0dd5-4544-8a92-318af40327ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.600105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7991106-0dd5-4544-8a92-318af40327ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.600852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7991106-0dd5-4544-8a92-318af40327ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.602793 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7991106-0dd5-4544-8a92-318af40327ae admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.605742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da550a1e87a148baa154dd5e0b03b11d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:30.613624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da550a1e87a148baa154dd5e0b03b11d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:30.675546 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da550a1e87a148baa154dd5e0b03b11d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:30.676379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da550a1e87a148baa154dd5e0b03b11d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:30.690141 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1017/2041] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:30 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:30.690989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7991106-0dd5-4544-8a92-318af40327ae admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:30.691196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da550a1e87a148baa154dd5e0b03b11d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:30.692642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c7991106-0dd5-4544-8a92-318af40327ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:30.701680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1025/2042] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:30 2022] DELETE /identity/v3/users/da550a1e87a148baa154dd5e0b03b11d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:30.702574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:30.704788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:30.768870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7991106-0dd5-4544-8a92-318af40327ae admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.769726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779dfcc-4dc2-41bc-ac36-d6581064943a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:30.774045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.774265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:30.776727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.776826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:30.776915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.776999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:30.777084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] PATH_INFO: `/v3/projects/12d945a3be564132969417814adecd60` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.777182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] PATH_INFO: `/v3/users/47c781420ca546c199914cf13efccdbe` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:30.777285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=12d945a3be564132969417814adecd60)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.777386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c781420ca546c199914cf13efccdbe)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:30.778408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.778533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:30.919834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:31.018399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c781420ca546c199914cf13efccdbe'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.018606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12d945a3be564132969417814adecd60', 'request_id': 'req-c55f6625-53f5-4198-94b6-421c17f04a0c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.024073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12d945a3be564132969417814adecd60', 'request_id': 'req-c55f6625-53f5-4198-94b6-421c17f04a0c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.026485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c781420ca546c199914cf13efccdbe'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.030022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:31.113019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12d945a3be564132969417814adecd60', 'request_id': 'req-c55f6625-53f5-4198-94b6-421c17f04a0c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.120411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] The token cache is being invalidate because project 12d945a3be564132969417814adecd60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:31.120698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.121348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.122861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1018/2043] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:30 2022] DELETE /identity/v3/projects/12d945a3be564132969417814adecd60 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:31.123602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.124993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:31.187063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f6625-53f5-4198-94b6-421c17f04a0c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.190992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c781420ca546c199914cf13efccdbe'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.191905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c781420ca546c199914cf13efccdbe'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.192019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ca6c5-e5c8-4d40-bd5d-6522e761667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.192550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ca6c5-e5c8-4d40-bd5d-6522e761667c admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.192836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ca6c5-e5c8-4d40-bd5d-6522e761667c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.193116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ca6c5-e5c8-4d40-bd5d-6522e761667c admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.193833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ca6c5-e5c8-4d40-bd5d-6522e761667c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.195798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ca6c5-e5c8-4d40-bd5d-6522e761667c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.205572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-144b5c1c-f68d-4e1b-a882-d08b45e00707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c781420ca546c199914cf13efccdbe'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.211390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1026/2044] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:30 2022] DELETE /identity/v3/users/47c781420ca546c199914cf13efccdbe => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:31.212628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdb0cb-418f-4025-9ac1-04765f7dc328 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.212918 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdb0cb-418f-4025-9ac1-04765f7dc328 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.213192 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdb0cb-418f-4025-9ac1-04765f7dc328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.232622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a6cdb0cb-418f-4025-9ac1-04765f7dc328 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:31.234039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a6cdb0cb-418f-4025-9ac1-04765f7dc328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:31.296669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1027/2045] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:31.298326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdb0cb-418f-4025-9ac1-04765f7dc328 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.298630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1019/2046] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:31 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:31.299672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ca6c5-e5c8-4d40-bd5d-6522e761667c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.301424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a6cdb0cb-418f-4025-9ac1-04765f7dc328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:31.301645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b05ca6c5-e5c8-4d40-bd5d-6522e761667c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:31.310743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:31.315535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdb0cb-418f-4025-9ac1-04765f7dc328 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.316113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ca6c5-e5c8-4d40-bd5d-6522e761667c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.323271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.323549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed71755-5bd1-442a-b04a-16c23e3e4222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.324507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed71755-5bd1-442a-b04a-16c23e3e4222 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.324968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed71755-5bd1-442a-b04a-16c23e3e4222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.325077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.325454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed71755-5bd1-442a-b04a-16c23e3e4222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.326412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.326412 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] PATH_INFO: `/v3/users/5417047080c545bfa31dbbcd9caa617b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.326674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed71755-5bd1-442a-b04a-16c23e3e4222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.329161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5417047080c545bfa31dbbcd9caa617b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.330134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed71755-5bd1-442a-b04a-16c23e3e4222 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.333147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.358239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1020/2047] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:31.358239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed71755-5bd1-442a-b04a-16c23e3e4222 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.358637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4ed71755-5bd1-442a-b04a-16c23e3e4222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:31.432712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed71755-5bd1-442a-b04a-16c23e3e4222 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.439916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67e167-fa70-4a3a-bbb9-4e7d6b4c4bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.440568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67e167-fa70-4a3a-bbb9-4e7d6b4c4bce service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.440874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67e167-fa70-4a3a-bbb9-4e7d6b4c4bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.441131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67e167-fa70-4a3a-bbb9-4e7d6b4c4bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.452089 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67e167-fa70-4a3a-bbb9-4e7d6b4c4bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.454060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67e167-fa70-4a3a-bbb9-4e7d6b4c4bce service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.464308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1021/2048] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:31.465413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67e167-fa70-4a3a-bbb9-4e7d6b4c4bce service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.475179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67e167-fa70-4a3a-bbb9-4e7d6b4c4bce service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.480768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8844b-5006-4836-9839-532e629ff977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.481477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8844b-5006-4836-9839-532e629ff977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.481728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8844b-5006-4836-9839-532e629ff977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.481995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8844b-5006-4836-9839-532e629ff977 admin admin] PATH_INFO: `/v3/projects/74887a5f67b14df79e9f22f63a79c8be/users/87f19140b8ff47808a5a83430e9efb88/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.499058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8844b-5006-4836-9839-532e629ff977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74887a5f67b14df79e9f22f63a79c8be, user_id=87f19140b8ff47808a5a83430e9efb88, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.502248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8844b-5006-4836-9839-532e629ff977 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.590533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5417047080c545bfa31dbbcd9caa617b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.599673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5417047080c545bfa31dbbcd9caa617b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.607838 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:31.615017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1022/2049] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:31 2022] PUT /identity/v3/projects/74887a5f67b14df79e9f22f63a79c8be/users/87f19140b8ff47808a5a83430e9efb88/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:31.618937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8844b-5006-4836-9839-532e629ff977 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.693549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5417047080c545bfa31dbbcd9caa617b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.695080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5417047080c545bfa31dbbcd9caa617b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.700918 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8844b-5006-4836-9839-532e629ff977 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.706585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949adea-02bb-47df-90c5-d4c6add02ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.707344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949adea-02bb-47df-90c5-d4c6add02ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.707629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949adea-02bb-47df-90c5-d4c6add02ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.707973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949adea-02bb-47df-90c5-d4c6add02ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.708732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949adea-02bb-47df-90c5-d4c6add02ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.710359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949adea-02bb-47df-90c5-d4c6add02ab2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.712424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5417047080c545bfa31dbbcd9caa617b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.720218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1028/2050] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:31 2022] DELETE /identity/v3/users/5417047080c545bfa31dbbcd9caa617b => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:31.721713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.726846 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1023/2051] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:31.727826 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949adea-02bb-47df-90c5-d4c6add02ab2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:31.730862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b949adea-02bb-47df-90c5-d4c6add02ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:31.734828 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:31.748576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949adea-02bb-47df-90c5-d4c6add02ab2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.816196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd470391-109a-40cf-a8f9-2a5ad7226ed5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:31.821428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.822205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.822595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.823313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] PATH_INFO: `/v3/projects/5f71edbe0d564667a9828ba045eb23e8` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.824888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f71edbe0d564667a9828ba045eb23e8)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.827595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.830719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d0d6-8ae1-4510-a196-e9aa360e789d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:31.831816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d0d6-8ae1-4510-a196-e9aa360e789d admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.834431 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d0d6-8ae1-4510-a196-e9aa360e789d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.834431 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d0d6-8ae1-4510-a196-e9aa360e789d admin admin] PATH_INFO: `/v3/projects/74887a5f67b14df79e9f22f63a79c8be/users/87f19140b8ff47808a5a83430e9efb88/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.835517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196d0d6-8ae1-4510-a196-e9aa360e789d admin admin] RBAC: Authorizing `identity:create_grant(project_id=74887a5f67b14df79e9f22f63a79c8be, user_id=87f19140b8ff47808a5a83430e9efb88, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:31.839079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196d0d6-8ae1-4510-a196-e9aa360e789d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:31.853283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1024/2052] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:31 2022] PUT /identity/v3/projects/74887a5f67b14df79e9f22f63a79c8be/users/87f19140b8ff47808a5a83430e9efb88/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:31.870237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c926-2c54-4010-8cc0-6b92b72ab4a0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:31.870533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c926-2c54-4010-8cc0-6b92b72ab4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:31.870932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c926-2c54-4010-8cc0-6b92b72ab4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:31.907015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2931c926-2c54-4010-8cc0-6b92b72ab4a0 None None] MFA Rules not processed for user `87f19140b8ff47808a5a83430e9efb88`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:31.909640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2931c926-2c54-4010-8cc0-6b92b72ab4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:31.974703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:31.981632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f71edbe0d564667a9828ba045eb23e8', 'request_id': 'req-93c20ddf-5229-4696-99f2-6bdb51787d5f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.989383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f71edbe0d564667a9828ba045eb23e8', 'request_id': 'req-93c20ddf-5229-4696-99f2-6bdb51787d5f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:31.998906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1025/2053] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:32.013120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c926-2c54-4010-8cc0-6b92b72ab4a0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.014410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2931c926-2c54-4010-8cc0-6b92b72ab4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.015345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f71edbe0d564667a9828ba045eb23e8', 'request_id': 'req-93c20ddf-5229-4696-99f2-6bdb51787d5f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.072619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c926-2c54-4010-8cc0-6b92b72ab4a0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.118700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] The token cache is being invalidate because project 5f71edbe0d564667a9828ba045eb23e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:32.119607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.120023 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.121835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1029/2054] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:31 2022] DELETE /identity/v3/projects/5f71edbe0d564667a9828ba045eb23e8 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:32.125461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff8941-da10-4b26-8755-068d48a38508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.126910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.127043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff8941-da10-4b26-8755-068d48a38508 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.127337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff8941-da10-4b26-8755-068d48a38508 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.127724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff8941-da10-4b26-8755-068d48a38508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.128582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.129688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d2ff8941-da10-4b26-8755-068d48a38508 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.196816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c20ddf-5229-4696-99f2-6bdb51787d5f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.200740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff8941-da10-4b26-8755-068d48a38508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.201622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.202270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.202557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.203373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] PATH_INFO: `/v3/projects/6585ae3af94e4984aa2a9c4b33bd266a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.203608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff8941-da10-4b26-8755-068d48a38508 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.204801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6585ae3af94e4984aa2a9c4b33bd266a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.207053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.214073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1026/2055] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:32.217324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817d13e-5be9-4b72-af9a-2ee914e6ab99 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.217691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817d13e-5be9-4b72-af9a-2ee914e6ab99 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.218045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817d13e-5be9-4b72-af9a-2ee914e6ab99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.239886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d817d13e-5be9-4b72-af9a-2ee914e6ab99 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:32.241118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d817d13e-5be9-4b72-af9a-2ee914e6ab99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.249169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1027/2056] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:32.258078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e925daf-8ac7-4ff5-b191-d0848feb1966 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.258363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e925daf-8ac7-4ff5-b191-d0848feb1966 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.258640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e925daf-8ac7-4ff5-b191-d0848feb1966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.277484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2e925daf-8ac7-4ff5-b191-d0848feb1966 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:32.278663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2e925daf-8ac7-4ff5-b191-d0848feb1966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.286892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1028/2057] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:32.291021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e925daf-8ac7-4ff5-b191-d0848feb1966 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.300544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e925daf-8ac7-4ff5-b191-d0848feb1966 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.305228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61059a-8df9-4a46-b29d-cfa6d94869c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.305825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61059a-8df9-4a46-b29d-cfa6d94869c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.306108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61059a-8df9-4a46-b29d-cfa6d94869c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.306386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61059a-8df9-4a46-b29d-cfa6d94869c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.307076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61059a-8df9-4a46-b29d-cfa6d94869c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:32.307384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61059a-8df9-4a46-b29d-cfa6d94869c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.309078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61059a-8df9-4a46-b29d-cfa6d94869c4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.322700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1029/2058] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:23:32.328229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61059a-8df9-4a46-b29d-cfa6d94869c4 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.340385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61059a-8df9-4a46-b29d-cfa6d94869c4 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.358122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6585ae3af94e4984aa2a9c4b33bd266a', 'request_id': 'req-cf1f53ef-6754-4e8f-97b6-ad648e24affb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.374860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6585ae3af94e4984aa2a9c4b33bd266a', 'request_id': 'req-cf1f53ef-6754-4e8f-97b6-ad648e24affb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.383277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6585ae3af94e4984aa2a9c4b33bd266a', 'request_id': 'req-cf1f53ef-6754-4e8f-97b6-ad648e24affb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.408514 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a580e-fe57-40f2-988c-fc4fc2a2d791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.409146 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a580e-fe57-40f2-988c-fc4fc2a2d791 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.409455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a580e-fe57-40f2-988c-fc4fc2a2d791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.409719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a580e-fe57-40f2-988c-fc4fc2a2d791 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.410387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a580e-fe57-40f2-988c-fc4fc2a2d791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.412451 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a580e-fe57-40f2-988c-fc4fc2a2d791 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.416811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:32.444878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] The token cache is being invalidate because project 6585ae3af94e4984aa2a9c4b33bd266a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:32.445130 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.445788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.447370 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1030/2059] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:32 2022] DELETE /identity/v3/projects/6585ae3af94e4984aa2a9c4b33bd266a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:32.448153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.449541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.463165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f53ef-6754-4e8f-97b6-ad648e24affb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.467861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.468488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.468769 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.469042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] PATH_INFO: `/v3/projects/e01602271f974f97b67562f35983a29a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.470324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01602271f974f97b67562f35983a29a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.472383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.528322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1030/2060] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:32 2022] POST /identity/v3/projects => generated 371 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:32.529583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba72e7-af1f-4d62-9f8a-0ee5bc81259b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.529869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba72e7-af1f-4d62-9f8a-0ee5bc81259b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.530138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba72e7-af1f-4d62-9f8a-0ee5bc81259b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.548336 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-29ba72e7-af1f-4d62-9f8a-0ee5bc81259b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:32.549544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-29ba72e7-af1f-4d62-9f8a-0ee5bc81259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.612876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1031/2061] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:32.613873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba72e7-af1f-4d62-9f8a-0ee5bc81259b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.629990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-29ba72e7-af1f-4d62-9f8a-0ee5bc81259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.630094 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01602271f974f97b67562f35983a29a', 'request_id': 'req-79edb60e-cbbc-49a9-a123-b5ff074aa53f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.647390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01602271f974f97b67562f35983a29a', 'request_id': 'req-79edb60e-cbbc-49a9-a123-b5ff074aa53f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.647523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba72e7-af1f-4d62-9f8a-0ee5bc81259b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.647849 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01602271f974f97b67562f35983a29a', 'request_id': 'req-79edb60e-cbbc-49a9-a123-b5ff074aa53f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.695500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5a91c-8da3-436b-adad-ea7e72180bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.696109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a91c-8da3-436b-adad-ea7e72180bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.696391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a91c-8da3-436b-adad-ea7e72180bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.696664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a91c-8da3-436b-adad-ea7e72180bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.697321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5a91c-8da3-436b-adad-ea7e72180bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.699236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5a91c-8da3-436b-adad-ea7e72180bd2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.726845 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] The token cache is being invalidate because project e01602271f974f97b67562f35983a29a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:32.727138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.727823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-79edb60e-cbbc-49a9-a123-b5ff074aa53f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:32.730277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1031/2062] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:32 2022] DELETE /identity/v3/projects/e01602271f974f97b67562f35983a29a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:32.731519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a74e3-c18e-404e-bc93-b2c69dcaed6f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.731827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a74e3-c18e-404e-bc93-b2c69dcaed6f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.732118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a74e3-c18e-404e-bc93-b2c69dcaed6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.753804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-103a74e3-c18e-404e-bc93-b2c69dcaed6f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:32.755045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-103a74e3-c18e-404e-bc93-b2c69dcaed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.763442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1032/2063] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:32.764820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fcc5f-d428-4c8b-9c16-907984a4ab5a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.765107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fcc5f-d428-4c8b-9c16-907984a4ab5a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.765386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fcc5f-d428-4c8b-9c16-907984a4ab5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.784465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-da7fcc5f-d428-4c8b-9c16-907984a4ab5a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:32.785626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-da7fcc5f-d428-4c8b-9c16-907984a4ab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.793385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1033/2064] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:32.794141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fcc5f-d428-4c8b-9c16-907984a4ab5a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.800550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1032/2065] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:32 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:32.801734 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ace1e-a23e-411b-b07e-938add07a572 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.802050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ace1e-a23e-411b-b07e-938add07a572 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.802344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ace1e-a23e-411b-b07e-938add07a572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.812752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fcc5f-d428-4c8b-9c16-907984a4ab5a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.817784 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:32.820368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01658-a805-49c1-8d4e-f5cb85f0f3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.820982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01658-a805-49c1-8d4e-f5cb85f0f3ff admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.821355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01658-a805-49c1-8d4e-f5cb85f0f3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.821685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01658-a805-49c1-8d4e-f5cb85f0f3ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.822434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e01658-a805-49c1-8d4e-f5cb85f0f3ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:32.822798 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e01658-a805-49c1-8d4e-f5cb85f0f3ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.830615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5f7ace1e-a23e-411b-b07e-938add07a572 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:32.830719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e01658-a805-49c1-8d4e-f5cb85f0f3ff admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.830805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5f7ace1e-a23e-411b-b07e-938add07a572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.833901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1033/2066] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:32.834892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ace1e-a23e-411b-b07e-938add07a572 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.836175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5f7ace1e-a23e-411b-b07e-938add07a572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:32.847972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1034/2067] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:32.847972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01658-a805-49c1-8d4e-f5cb85f0f3ff admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.848422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ace1e-a23e-411b-b07e-938add07a572 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.854711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f0aa-86c1-463f-add0-4552320c3556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.855603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f0aa-86c1-463f-add0-4552320c3556 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.856236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f0aa-86c1-463f-add0-4552320c3556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.856236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f0aa-86c1-463f-add0-4552320c3556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.856614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01658-a805-49c1-8d4e-f5cb85f0f3ff admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.856940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6f0aa-86c1-463f-add0-4552320c3556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.858630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6f0aa-86c1-463f-add0-4552320c3556 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.862234 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64240b4-1004-4780-bff1-6487aa29ea2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.863015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64240b4-1004-4780-bff1-6487aa29ea2d admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.863482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64240b4-1004-4780-bff1-6487aa29ea2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.864168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64240b4-1004-4780-bff1-6487aa29ea2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.865009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64240b4-1004-4780-bff1-6487aa29ea2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:32.865329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64240b4-1004-4780-bff1-6487aa29ea2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.866974 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64240b4-1004-4780-bff1-6487aa29ea2d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.870308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1034/2068] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:32.871310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f0aa-86c1-463f-add0-4552320c3556 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.888228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:32.888228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f0aa-86c1-463f-add0-4552320c3556 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.888397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1035/2069] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:32.889085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64240b4-1004-4780-bff1-6487aa29ea2d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.893579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276c13c-9ccf-46ae-bb83-c3cc181731e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.894390 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276c13c-9ccf-46ae-bb83-c3cc181731e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.894645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276c13c-9ccf-46ae-bb83-c3cc181731e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.894986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276c13c-9ccf-46ae-bb83-c3cc181731e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.896178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276c13c-9ccf-46ae-bb83-c3cc181731e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.899873 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64240b4-1004-4780-bff1-6487aa29ea2d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.900157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276c13c-9ccf-46ae-bb83-c3cc181731e3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.905555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe93b87-fcb3-4a64-9b7d-d4f58b35eb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.906159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe93b87-fcb3-4a64-9b7d-d4f58b35eb78 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.906551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe93b87-fcb3-4a64-9b7d-d4f58b35eb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.906713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe93b87-fcb3-4a64-9b7d-d4f58b35eb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.907416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe93b87-fcb3-4a64-9b7d-d4f58b35eb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.909111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe93b87-fcb3-4a64-9b7d-d4f58b35eb78 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:32.919867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1036/2070] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:32.920762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe93b87-fcb3-4a64-9b7d-d4f58b35eb78 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:32.936647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe93b87-fcb3-4a64-9b7d-d4f58b35eb78 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:32.938439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84333e1b-072b-444f-9d8d-a50689e24c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:32.939054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84333e1b-072b-444f-9d8d-a50689e24c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:32.939334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84333e1b-072b-444f-9d8d-a50689e24c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:32.939981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84333e1b-072b-444f-9d8d-a50689e24c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:32.940714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84333e1b-072b-444f-9d8d-a50689e24c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:32.943955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84333e1b-072b-444f-9d8d-a50689e24c2f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.000296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1035/2071] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:32 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:33.000296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276c13c-9ccf-46ae-bb83-c3cc181731e3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.067596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1037/2072] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:32 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:33.069672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276c13c-9ccf-46ae-bb83-c3cc181731e3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.071442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84333e1b-072b-444f-9d8d-a50689e24c2f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.133083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:33.144490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf01de0-067e-423b-9209-d8e551ed4c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.145268 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01de0-067e-423b-9209-d8e551ed4c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.145548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01de0-067e-423b-9209-d8e551ed4c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.145822 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf01de0-067e-423b-9209-d8e551ed4c1a admin admin] PATH_INFO: `/v3/projects/2129d74b666f4144af582be0e24e2027/users/fa8f8aea6be94cd2bf47f18cc0116364/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.151385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84333e1b-072b-444f-9d8d-a50689e24c2f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.157120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78607de6-1c7b-4289-ad7b-95daf98dfcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.157757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78607de6-1c7b-4289-ad7b-95daf98dfcf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.158050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78607de6-1c7b-4289-ad7b-95daf98dfcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.158312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78607de6-1c7b-4289-ad7b-95daf98dfcf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.158999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78607de6-1c7b-4289-ad7b-95daf98dfcf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.161500 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78607de6-1c7b-4289-ad7b-95daf98dfcf8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.165242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf01de0-067e-423b-9209-d8e551ed4c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2129d74b666f4144af582be0e24e2027, user_id=fa8f8aea6be94cd2bf47f18cc0116364, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.168107 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf01de0-067e-423b-9209-d8e551ed4c1a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.464143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1038/2073] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:33 2022] POST /identity/v3/users => generated 287 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:33.464143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78607de6-1c7b-4289-ad7b-95daf98dfcf8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.464556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1036/2074] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:32 2022] PUT /identity/v3/projects/2129d74b666f4144af582be0e24e2027/users/fa8f8aea6be94cd2bf47f18cc0116364/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:33.464556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf01de0-067e-423b-9209-d8e551ed4c1a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.466477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-acf01de0-067e-423b-9209-d8e551ed4c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:33.475538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78607de6-1c7b-4289-ad7b-95daf98dfcf8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.512191 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:33.533015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf01de0-067e-423b-9209-d8e551ed4c1a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.537564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.538211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.538561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.538689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2eb2d6-ad32-4aa0-9a4d-be423ad938fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.538840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] PATH_INFO: `/v3/users/81880947f409400c93e1694fab6e4d67` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.539231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2eb2d6-ad32-4aa0-9a4d-be423ad938fc admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.539530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2eb2d6-ad32-4aa0-9a4d-be423ad938fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.539815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2eb2d6-ad32-4aa0-9a4d-be423ad938fc admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.542738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81880947f409400c93e1694fab6e4d67)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.542853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2eb2d6-ad32-4aa0-9a4d-be423ad938fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.542956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.543048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2eb2d6-ad32-4aa0-9a4d-be423ad938fc admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.624267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1039/2075] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:33 2022] POST /identity/v3/users => generated 283 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:33.625163 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2eb2d6-ad32-4aa0-9a4d-be423ad938fc admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.634618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2eb2d6-ad32-4aa0-9a4d-be423ad938fc admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.639374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809642c6-53e2-4496-a131-f9dbf116243c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.640008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809642c6-53e2-4496-a131-f9dbf116243c admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.640837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809642c6-53e2-4496-a131-f9dbf116243c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.641111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809642c6-53e2-4496-a131-f9dbf116243c admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.641801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809642c6-53e2-4496-a131-f9dbf116243c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.643421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809642c6-53e2-4496-a131-f9dbf116243c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.653272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1040/2076] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:33.654393 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fde65-cc6c-465d-95ed-125a04b40f56 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.654685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fde65-cc6c-465d-95ed-125a04b40f56 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.654964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fde65-cc6c-465d-95ed-125a04b40f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.660682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:33.673784 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-351fde65-cc6c-465d-95ed-125a04b40f56 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:33.674990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-351fde65-cc6c-465d-95ed-125a04b40f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:33.682668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1041/2077] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:33.683427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fde65-cc6c-465d-95ed-125a04b40f56 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.692964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fde65-cc6c-465d-95ed-125a04b40f56 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.697472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0af51-7dfa-4f4a-b238-682284dc08f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.698089 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0af51-7dfa-4f4a-b238-682284dc08f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.698379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0af51-7dfa-4f4a-b238-682284dc08f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.698655 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0af51-7dfa-4f4a-b238-682284dc08f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.699353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0af51-7dfa-4f4a-b238-682284dc08f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.700970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0af51-7dfa-4f4a-b238-682284dc08f9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.711466 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1042/2078] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:33.712281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0af51-7dfa-4f4a-b238-682284dc08f9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.725801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0af51-7dfa-4f4a-b238-682284dc08f9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.727700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da9d47-48b8-44fa-b78a-69fa8601036e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.728366 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da9d47-48b8-44fa-b78a-69fa8601036e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.728671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da9d47-48b8-44fa-b78a-69fa8601036e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.728944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da9d47-48b8-44fa-b78a-69fa8601036e admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.729661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da9d47-48b8-44fa-b78a-69fa8601036e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.731711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da9d47-48b8-44fa-b78a-69fa8601036e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.742162 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1043/2079] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:33.743005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da9d47-48b8-44fa-b78a-69fa8601036e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.746789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-22da9d47-48b8-44fa-b78a-69fa8601036e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:33.757620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81880947f409400c93e1694fab6e4d67'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:33.764277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81880947f409400c93e1694fab6e4d67'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:33.802579 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da9d47-48b8-44fa-b78a-69fa8601036e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.806251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8f2e1-1ba3-4a1c-ae12-ae5f3b0709d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.806863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8f2e1-1ba3-4a1c-ae12-ae5f3b0709d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.807145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8f2e1-1ba3-4a1c-ae12-ae5f3b0709d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.807423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8f2e1-1ba3-4a1c-ae12-ae5f3b0709d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.816321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8f2e1-1ba3-4a1c-ae12-ae5f3b0709d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.817738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8f2e1-1ba3-4a1c-ae12-ae5f3b0709d9 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.826851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1044/2080] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:33.827621 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8f2e1-1ba3-4a1c-ae12-ae5f3b0709d9 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.830536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81880947f409400c93e1694fab6e4d67'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:33.831162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81880947f409400c93e1694fab6e4d67'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:33.831770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81880947f409400c93e1694fab6e4d67'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:33.838097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8f2e1-1ba3-4a1c-ae12-ae5f3b0709d9 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.841766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b35353-cc75-4053-9576-d7a7f8c14851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.842733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b35353-cc75-4053-9576-d7a7f8c14851 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.842733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b35353-cc75-4053-9576-d7a7f8c14851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.842945 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b35353-cc75-4053-9576-d7a7f8c14851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.844354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b35353-cc75-4053-9576-d7a7f8c14851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.845945 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b35353-cc75-4053-9576-d7a7f8c14851 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.852872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1037/2081] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:33 2022] DELETE /identity/v3/users/81880947f409400c93e1694fab6e4d67 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:33.853681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.855248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:33.859701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1045/2082] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:33.860974 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b3ddb-ad66-4e3c-ac5b-69642809f076 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.861268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b3ddb-ad66-4e3c-ac5b-69642809f076 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.861671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b3ddb-ad66-4e3c-ac5b-69642809f076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.880757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f53b3ddb-ad66-4e3c-ac5b-69642809f076 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:33.881991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f53b3ddb-ad66-4e3c-ac5b-69642809f076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:33.919753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfd86c-55d6-4c17-ba8d-0d9436d6d400 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.925140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b7a66-bcdf-451e-b828-33408eabfc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.931676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b7a66-bcdf-451e-b828-33408eabfc1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.931676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b7a66-bcdf-451e-b828-33408eabfc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.931676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b7a66-bcdf-451e-b828-33408eabfc1d admin admin] PATH_INFO: `/v3/projects/64a2eddfc6494005aeda51bec7fb256b/users/c8fd5f27c0a742239e1e5851c92dd30f/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.945648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b7a66-bcdf-451e-b828-33408eabfc1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a2eddfc6494005aeda51bec7fb256b, user_id=c8fd5f27c0a742239e1e5851c92dd30f, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.946248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1046/2083] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:33.947086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b3ddb-ad66-4e3c-ac5b-69642809f076 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:33.949483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b7a66-bcdf-451e-b828-33408eabfc1d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:33.951091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f53b3ddb-ad66-4e3c-ac5b-69642809f076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:33.964739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b3ddb-ad66-4e3c-ac5b-69642809f076 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:33.971321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6fbcb-cd16-420b-978f-b83b4d6f4346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:33.972262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6fbcb-cd16-420b-978f-b83b4d6f4346 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:33.972610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6fbcb-cd16-420b-978f-b83b4d6f4346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:33.973003 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6fbcb-cd16-420b-978f-b83b4d6f4346 admin admin] PATH_INFO: `/v3/projects/cf92482579224b5abbbe96c35210f19f/users/0f5eeb67fbe84d38827cc4520ac1b423/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:33.994324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6fbcb-cd16-420b-978f-b83b4d6f4346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf92482579224b5abbbe96c35210f19f, user_id=0f5eeb67fbe84d38827cc4520ac1b423, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:33.997847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6fbcb-cd16-420b-978f-b83b4d6f4346 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:34.036400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1038/2084] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:33 2022] PUT /identity/v3/projects/64a2eddfc6494005aeda51bec7fb256b/users/c8fd5f27c0a742239e1e5851c92dd30f/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:34.037414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b7a66-bcdf-451e-b828-33408eabfc1d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:34.039125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bf0b7a66-bcdf-451e-b828-33408eabfc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:34.053026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:34.055281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:34.061678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1047/2085] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:33 2022] PUT /identity/v3/projects/cf92482579224b5abbbe96c35210f19f/users/0f5eeb67fbe84d38827cc4520ac1b423/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:34.072398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afbb79-e4a3-4dbd-b73f-957f4ced36bb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:34.072398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afbb79-e4a3-4dbd-b73f-957f4ced36bb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:34.072398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afbb79-e4a3-4dbd-b73f-957f4ced36bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:34.095528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e3afbb79-e4a3-4dbd-b73f-957f4ced36bb None None] MFA Rules not processed for user `c8fd5f27c0a742239e1e5851c92dd30f`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:34.096900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e3afbb79-e4a3-4dbd-b73f-957f4ced36bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:34.103790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b7a66-bcdf-451e-b828-33408eabfc1d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:34.173435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28bce6-685f-450f-94b4-733c7dd2d23e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:34.173570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1048/2086] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:34.174160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28bce6-685f-450f-94b4-733c7dd2d23e service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:34.174444 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28bce6-685f-450f-94b4-733c7dd2d23e service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:34.174722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28bce6-685f-450f-94b4-733c7dd2d23e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:34.174831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952c5bf-0dc0-425a-9e28-62df7e153baa None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:34.175072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952c5bf-0dc0-425a-9e28-62df7e153baa None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:34.175425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952c5bf-0dc0-425a-9e28-62df7e153baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:34.206794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3952c5bf-0dc0-425a-9e28-62df7e153baa None None] MFA Rules not processed for user `0f5eeb67fbe84d38827cc4520ac1b423`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:34.208195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3952c5bf-0dc0-425a-9e28-62df7e153baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:34.245471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28bce6-685f-450f-94b4-733c7dd2d23e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:34.247047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28bce6-685f-450f-94b4-733c7dd2d23e service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:34.274377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1039/2087] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:34.275249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28bce6-685f-450f-94b4-733c7dd2d23e service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:34.277938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cb28bce6-685f-450f-94b4-733c7dd2d23e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:34.308697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1049/2088] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:34.322633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952c5bf-0dc0-425a-9e28-62df7e153baa None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:34.324452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3952c5bf-0dc0-425a-9e28-62df7e153baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:34.358662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28bce6-685f-450f-94b4-733c7dd2d23e service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:34.363309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe52aac-306f-452e-b352-d4c0ffe8c21e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:34.363969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe52aac-306f-452e-b352-d4c0ffe8c21e service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:34.364252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe52aac-306f-452e-b352-d4c0ffe8c21e service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:34.364558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe52aac-306f-452e-b352-d4c0ffe8c21e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:34.383345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe52aac-306f-452e-b352-d4c0ffe8c21e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:34.385089 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe52aac-306f-452e-b352-d4c0ffe8c21e service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:34.396066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1040/2089] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:34.401953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952c5bf-0dc0-425a-9e28-62df7e153baa None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:34.405914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6c428-5b1d-403f-ad83-46357af0696c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a234a96eccd4f38b0213d7d0461ad24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:34.411108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6c428-5b1d-403f-ad83-46357af0696c service glance] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:34.411108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6c428-5b1d-403f-ad83-46357af0696c service glance] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:34.411108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6c428-5b1d-403f-ad83-46357af0696c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:34.415806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6c428-5b1d-403f-ad83-46357af0696c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:34.417256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6c428-5b1d-403f-ad83-46357af0696c service glance] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:34.438636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1050/2090] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:34.881130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe52aac-306f-452e-b352-d4c0ffe8c21e service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:34.883554 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8fe52aac-306f-452e-b352-d4c0ffe8c21e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:34.899937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe52aac-306f-452e-b352-d4c0ffe8c21e service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:34.907495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:34.907495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:34.907495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:34.907495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] PATH_INFO: `/v3/projects/b6efdb494bf9472ebbd10ba0bbc9c13c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:34.912122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6efdb494bf9472ebbd10ba0bbc9c13c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:34.912122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:35.018129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:35.040531 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6efdb494bf9472ebbd10ba0bbc9c13c', 'request_id': 'req-b254afc6-bbed-464b-9e39-f339da7385c1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:35.050147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6efdb494bf9472ebbd10ba0bbc9c13c', 'request_id': 'req-b254afc6-bbed-464b-9e39-f339da7385c1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:35.126880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6efdb494bf9472ebbd10ba0bbc9c13c', 'request_id': 'req-b254afc6-bbed-464b-9e39-f339da7385c1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:35.139890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] The token cache is being invalidate because project b6efdb494bf9472ebbd10ba0bbc9c13c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:35.140821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:35.141543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b254afc6-bbed-464b-9e39-f339da7385c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:35.143153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1041/2091] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:34 2022] DELETE /identity/v3/projects/b6efdb494bf9472ebbd10ba0bbc9c13c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:35.150220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6c428-5b1d-403f-ad83-46357af0696c service glance] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:35.162074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6c428-5b1d-403f-ad83-46357af0696c service glance] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:35.173684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:35.222887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:35.223533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:35.223846 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:35.224128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] PATH_INFO: `/v3/users/87f19140b8ff47808a5a83430e9efb88` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:35.225783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f19140b8ff47808a5a83430e9efb88)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:35.227535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:35.464006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f19140b8ff47808a5a83430e9efb88'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:35.490062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f19140b8ff47808a5a83430e9efb88'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:35.533401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4194146-f62f-4d00-8252-6979062e24f2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:35.533852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4194146-f62f-4d00-8252-6979062e24f2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:35.534135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4194146-f62f-4d00-8252-6979062e24f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:35.568698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d4194146-f62f-4d00-8252-6979062e24f2 None None] MFA Rules not processed for user `fa8f8aea6be94cd2bf47f18cc0116364`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:35.569973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d4194146-f62f-4d00-8252-6979062e24f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:35.621624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f19140b8ff47808a5a83430e9efb88'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:35.622353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f19140b8ff47808a5a83430e9efb88'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:35.638196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f19140b8ff47808a5a83430e9efb88'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:35.647231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1051/2092] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:35 2022] DELETE /identity/v3/users/87f19140b8ff47808a5a83430e9efb88 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:35.676910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1042/2093] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:35.677747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:35.679527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:35.749894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01c17b-3b39-4e4a-831c-4cb86e255f97 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:35.755150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c57d5-7b1d-4284-aab8-440f6f0afc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:35.755797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c57d5-7b1d-4284-aab8-440f6f0afc47 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:35.756387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c57d5-7b1d-4284-aab8-440f6f0afc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:35.756503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c57d5-7b1d-4284-aab8-440f6f0afc47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:35.757202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c57d5-7b1d-4284-aab8-440f6f0afc47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:35.759465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c57d5-7b1d-4284-aab8-440f6f0afc47 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:35.856120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1052/2094] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:35 2022] POST /identity/v3/projects => generated 373 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:35.860875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4194146-f62f-4d00-8252-6979062e24f2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:35.871538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4194146-f62f-4d00-8252-6979062e24f2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:35.928409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79f69d-2882-47ea-a92f-eea17fbc6040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:35.929130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79f69d-2882-47ea-a92f-eea17fbc6040 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:35.929490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79f69d-2882-47ea-a92f-eea17fbc6040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:35.929833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79f69d-2882-47ea-a92f-eea17fbc6040 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:35.930522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79f69d-2882-47ea-a92f-eea17fbc6040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:35.932502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79f69d-2882-47ea-a92f-eea17fbc6040 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.024167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1043/2095] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:35 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:36.028188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c57d5-7b1d-4284-aab8-440f6f0afc47 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.039292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c57d5-7b1d-4284-aab8-440f6f0afc47 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.043906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c028c8-b0e8-40e4-8ccb-68697c9cf8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.044505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c028c8-b0e8-40e4-8ccb-68697c9cf8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.044784 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c028c8-b0e8-40e4-8ccb-68697c9cf8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.045059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c028c8-b0e8-40e4-8ccb-68697c9cf8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.045744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c028c8-b0e8-40e4-8ccb-68697c9cf8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.047877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c028c8-b0e8-40e4-8ccb-68697c9cf8f6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.058186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1053/2096] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:36.062769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79f69d-2882-47ea-a92f-eea17fbc6040 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.073208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79f69d-2882-47ea-a92f-eea17fbc6040 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.082155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001fa95-39e4-4e72-b2c7-602d3f680219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.082155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001fa95-39e4-4e72-b2c7-602d3f680219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.082155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001fa95-39e4-4e72-b2c7-602d3f680219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.082155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001fa95-39e4-4e72-b2c7-602d3f680219 admin admin] PATH_INFO: `/v3/projects/ffacd75c314e42bdb90754176d3f0a7f/users/eb2a8b66a4ef452ab3a84910841155aa/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.083360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c028c8-b0e8-40e4-8ccb-68697c9cf8f6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.086011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-27c028c8-b0e8-40e4-8ccb-68697c9cf8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:36.099383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001fa95-39e4-4e72-b2c7-602d3f680219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffacd75c314e42bdb90754176d3f0a7f, user_id=eb2a8b66a4ef452ab3a84910841155aa, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.101419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c028c8-b0e8-40e4-8ccb-68697c9cf8f6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.107051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001fa95-39e4-4e72-b2c7-602d3f680219 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.107252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.107650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:36.108415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.108806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.109703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] PATH_INFO: `/v3/projects/74887a5f67b14df79e9f22f63a79c8be` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.110921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74887a5f67b14df79e9f22f63a79c8be)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.112889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.115606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:36.177592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1044/2097] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:36 2022] PUT /identity/v3/projects/ffacd75c314e42bdb90754176d3f0a7f/users/eb2a8b66a4ef452ab3a84910841155aa/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:36.181347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001fa95-39e4-4e72-b2c7-602d3f680219 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.213289 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74887a5f67b14df79e9f22f63a79c8be', 'request_id': 'req-c341d119-6edf-4355-99d1-d59d1eba6452'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:36.220230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74887a5f67b14df79e9f22f63a79c8be', 'request_id': 'req-c341d119-6edf-4355-99d1-d59d1eba6452'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:36.225773 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74887a5f67b14df79e9f22f63a79c8be', 'request_id': 'req-c341d119-6edf-4355-99d1-d59d1eba6452'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:36.244410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001fa95-39e4-4e72-b2c7-602d3f680219 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.299655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a7991-1ac4-4841-a380-51521822e99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.300325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a7991-1ac4-4841-a380-51521822e99c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.300610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a7991-1ac4-4841-a380-51521822e99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.300888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a7991-1ac4-4841-a380-51521822e99c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.301582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a7991-1ac4-4841-a380-51521822e99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.303310 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a7991-1ac4-4841-a380-51521822e99c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.304924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] The token cache is being invalidate because project 74887a5f67b14df79e9f22f63a79c8be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:36.305228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:36.305870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c341d119-6edf-4355-99d1-d59d1eba6452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:36.307413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1054/2098] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:36 2022] DELETE /identity/v3/projects/74887a5f67b14df79e9f22f63a79c8be => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:36.312294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40d535-63fb-4d87-96a6-45f46cd86f28 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.312585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40d535-63fb-4d87-96a6-45f46cd86f28 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.313069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40d535-63fb-4d87-96a6-45f46cd86f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.317932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1045/2099] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:36.318884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a7991-1ac4-4841-a380-51521822e99c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.320517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-058a7991-1ac4-4841-a380-51521822e99c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:36.332401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a7991-1ac4-4841-a380-51521822e99c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.333237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1a40d535-63fb-4d87-96a6-45f46cd86f28 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:36.334430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1a40d535-63fb-4d87-96a6-45f46cd86f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:36.337511 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181cb76-c4ac-49f4-9d41-d2a1b2635ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.338182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181cb76-c4ac-49f4-9d41-d2a1b2635ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.338469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181cb76-c4ac-49f4-9d41-d2a1b2635ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.338753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181cb76-c4ac-49f4-9d41-d2a1b2635ee5 admin admin] PATH_INFO: `/v3/projects/ffacd75c314e42bdb90754176d3f0a7f/users/eb2a8b66a4ef452ab3a84910841155aa/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.340869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181cb76-c4ac-49f4-9d41-d2a1b2635ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffacd75c314e42bdb90754176d3f0a7f, user_id=eb2a8b66a4ef452ab3a84910841155aa, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.342425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1055/2100] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:36.344128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181cb76-c4ac-49f4-9d41-d2a1b2635ee5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.350287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aad29c-95bb-4bac-9717-9243bfd19732 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.350577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aad29c-95bb-4bac-9717-9243bfd19732 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.350852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aad29c-95bb-4bac-9717-9243bfd19732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.355215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1046/2101] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:36 2022] PUT /identity/v3/projects/ffacd75c314e42bdb90754176d3f0a7f/users/eb2a8b66a4ef452ab3a84910841155aa/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:36.371261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a7aad29c-95bb-4bac-9717-9243bfd19732 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:36.372491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a7aad29c-95bb-4bac-9717-9243bfd19732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:36.433134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1056/2102] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:36.437470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181cb76-c4ac-49f4-9d41-d2a1b2635ee5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.449631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181cb76-c4ac-49f4-9d41-d2a1b2635ee5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.454972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5dcbd-30c2-46b9-861c-8d529fb523a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.455484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5dcbd-30c2-46b9-861c-8d529fb523a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.455901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5dcbd-30c2-46b9-861c-8d529fb523a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.456067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5dcbd-30c2-46b9-861c-8d529fb523a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.456757 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5dcbd-30c2-46b9-861c-8d529fb523a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:36.457115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5dcbd-30c2-46b9-861c-8d529fb523a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.458812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5dcbd-30c2-46b9-861c-8d529fb523a9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.481425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1047/2103] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:36.483197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aad29c-95bb-4bac-9717-9243bfd19732 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.493875 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aad29c-95bb-4bac-9717-9243bfd19732 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.516564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95b5b8-8309-41d1-92cf-404a642201fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.517361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95b5b8-8309-41d1-92cf-404a642201fe admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.517761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95b5b8-8309-41d1-92cf-404a642201fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.518694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95b5b8-8309-41d1-92cf-404a642201fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.519485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95b5b8-8309-41d1-92cf-404a642201fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.526597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95b5b8-8309-41d1-92cf-404a642201fe admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.590337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1057/2104] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:36 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:36.591334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5dcbd-30c2-46b9-861c-8d529fb523a9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.601906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5dcbd-30c2-46b9-861c-8d529fb523a9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.685078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771d952-561e-42ba-9fcd-c07d11920db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.685714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771d952-561e-42ba-9fcd-c07d11920db6 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.685996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771d952-561e-42ba-9fcd-c07d11920db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.686272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771d952-561e-42ba-9fcd-c07d11920db6 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.686952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771d952-561e-42ba-9fcd-c07d11920db6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.689029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771d952-561e-42ba-9fcd-c07d11920db6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.776907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1048/2105] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:36 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:36.781529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95b5b8-8309-41d1-92cf-404a642201fe admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.792555 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95b5b8-8309-41d1-92cf-404a642201fe admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.798219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca646c8-10e2-45bb-b5ce-9d3b06fcd483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.798838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca646c8-10e2-45bb-b5ce-9d3b06fcd483 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.802826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca646c8-10e2-45bb-b5ce-9d3b06fcd483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.802826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca646c8-10e2-45bb-b5ce-9d3b06fcd483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.802826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca646c8-10e2-45bb-b5ce-9d3b06fcd483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.802826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca646c8-10e2-45bb-b5ce-9d3b06fcd483 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.813129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1058/2106] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:36.817741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771d952-561e-42ba-9fcd-c07d11920db6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.823351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:36.829334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771d952-561e-42ba-9fcd-c07d11920db6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.834290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1e620-0c31-4da3-acd3-8a2e0435b354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.834921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1e620-0c31-4da3-acd3-8a2e0435b354 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.835214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1e620-0c31-4da3-acd3-8a2e0435b354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.835482 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1e620-0c31-4da3-acd3-8a2e0435b354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.836344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1e620-0c31-4da3-acd3-8a2e0435b354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.837823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1e620-0c31-4da3-acd3-8a2e0435b354 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.854054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1049/2107] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:36.859332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca646c8-10e2-45bb-b5ce-9d3b06fcd483 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:36.871408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca646c8-10e2-45bb-b5ce-9d3b06fcd483 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:36.876967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1cfbe-ec90-482c-a78b-14bdae144121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:36.877670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1cfbe-ec90-482c-a78b-14bdae144121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:36.878015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1cfbe-ec90-482c-a78b-14bdae144121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:36.878367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1cfbe-ec90-482c-a78b-14bdae144121 admin admin] PATH_INFO: `/v3/projects/195d207cd1b547e8963ff59153c0c7dd/users/b9f420d878074704a9707b30df462302/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:36.894548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1cfbe-ec90-482c-a78b-14bdae144121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=195d207cd1b547e8963ff59153c0c7dd, user_id=b9f420d878074704a9707b30df462302, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:36.897479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1cfbe-ec90-482c-a78b-14bdae144121 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:36.988251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1059/2108] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:36 2022] PUT /identity/v3/projects/195d207cd1b547e8963ff59153c0c7dd/users/b9f420d878074704a9707b30df462302/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:37.000818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bba5b-2070-4062-b510-ea5491ed3689 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:37.001302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bba5b-2070-4062-b510-ea5491ed3689 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:37.001392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bba5b-2070-4062-b510-ea5491ed3689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:37.028499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-fb0bba5b-2070-4062-b510-ea5491ed3689 None None] MFA Rules not processed for user `b9f420d878074704a9707b30df462302`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:37.029733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fb0bba5b-2070-4062-b510-ea5491ed3689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:37.091593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1050/2109] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:37.096326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1cfbe-ec90-482c-a78b-14bdae144121 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:37.127895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:37.159437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1cfbe-ec90-482c-a78b-14bdae144121 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:37.164424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28061753-6cda-42aa-bbc0-e95442a90921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:37.165160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28061753-6cda-42aa-bbc0-e95442a90921 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:37.165445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28061753-6cda-42aa-bbc0-e95442a90921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:37.165718 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28061753-6cda-42aa-bbc0-e95442a90921 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:37.166401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28061753-6cda-42aa-bbc0-e95442a90921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:37.168348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28061753-6cda-42aa-bbc0-e95442a90921 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:37.254406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1060/2110] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:37 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:37.258538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bba5b-2070-4062-b510-ea5491ed3689 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:37.268940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bba5b-2070-4062-b510-ea5491ed3689 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:37.323414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f836cb-9779-4215-b9f5-bf2b64efec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:37.324089 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f836cb-9779-4215-b9f5-bf2b64efec49 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:37.324373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f836cb-9779-4215-b9f5-bf2b64efec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:37.324648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f836cb-9779-4215-b9f5-bf2b64efec49 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:37.325322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f836cb-9779-4215-b9f5-bf2b64efec49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:37.327237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f836cb-9779-4215-b9f5-bf2b64efec49 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:37.411103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1051/2111] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:37 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:37.415172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28061753-6cda-42aa-bbc0-e95442a90921 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:37.425644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28061753-6cda-42aa-bbc0-e95442a90921 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:37.430626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e17a-8da7-46b3-8bd6-364917ef9b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:37.431393 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e17a-8da7-46b3-8bd6-364917ef9b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:37.431679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e17a-8da7-46b3-8bd6-364917ef9b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:37.431991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e17a-8da7-46b3-8bd6-364917ef9b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:37.432729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6e17a-8da7-46b3-8bd6-364917ef9b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:37.434360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6e17a-8da7-46b3-8bd6-364917ef9b71 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:37.444961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1061/2112] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:37.448979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f836cb-9779-4215-b9f5-bf2b64efec49 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:37.462034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f836cb-9779-4215-b9f5-bf2b64efec49 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:37.467168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41ebee-16dc-4b10-bf10-635cb4821771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:37.467968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41ebee-16dc-4b10-bf10-635cb4821771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:37.468255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41ebee-16dc-4b10-bf10-635cb4821771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:37.468648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41ebee-16dc-4b10-bf10-635cb4821771 admin admin] PATH_INFO: `/v3/projects/6583d7b9e1e348d6a184e4fca5a7e2ae/users/d1a1b2503bb84f2a9a9fb722c6d52cbb/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:37.484511 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41ebee-16dc-4b10-bf10-635cb4821771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6583d7b9e1e348d6a184e4fca5a7e2ae, user_id=d1a1b2503bb84f2a9a9fb722c6d52cbb, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:37.487247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41ebee-16dc-4b10-bf10-635cb4821771 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:37.574932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1052/2113] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:37 2022] PUT /identity/v3/projects/6583d7b9e1e348d6a184e4fca5a7e2ae/users/d1a1b2503bb84f2a9a9fb722c6d52cbb/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:37.581155 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e17a-8da7-46b3-8bd6-364917ef9b71 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:37.601037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e17a-8da7-46b3-8bd6-364917ef9b71 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:37.668400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5016268-8126-4f54-808c-a61c57358d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:37.669027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5016268-8126-4f54-808c-a61c57358d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:37.669314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5016268-8126-4f54-808c-a61c57358d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:37.669592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5016268-8126-4f54-808c-a61c57358d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:37.670462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5016268-8126-4f54-808c-a61c57358d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:37.672695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5016268-8126-4f54-808c-a61c57358d12 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:37.686054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1062/2114] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:37.687135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41ebee-16dc-4b10-bf10-635cb4821771 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:37.704140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41ebee-16dc-4b10-bf10-635cb4821771 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:37.709519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37fc77-f21b-4853-8b0a-4b6e358c73c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:37.710194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37fc77-f21b-4853-8b0a-4b6e358c73c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:37.710402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e8b3-5ddd-4c10-9c32-e7a0c164e363 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:37.710521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37fc77-f21b-4853-8b0a-4b6e358c73c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:37.710700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e8b3-5ddd-4c10-9c32-e7a0c164e363 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:37.710811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37fc77-f21b-4853-8b0a-4b6e358c73c2 admin admin] PATH_INFO: `/v3/projects/6583d7b9e1e348d6a184e4fca5a7e2ae/users/d1a1b2503bb84f2a9a9fb722c6d52cbb/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:37.710987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e8b3-5ddd-4c10-9c32-e7a0c164e363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:37.712750 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37fc77-f21b-4853-8b0a-4b6e358c73c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6583d7b9e1e348d6a184e4fca5a7e2ae, user_id=d1a1b2503bb84f2a9a9fb722c6d52cbb, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:37.715820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37fc77-f21b-4853-8b0a-4b6e358c73c2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:37.749466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1053/2115] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:37 2022] PUT /identity/v3/projects/6583d7b9e1e348d6a184e4fca5a7e2ae/users/d1a1b2503bb84f2a9a9fb722c6d52cbb/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:37.764554 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d084d1-78d8-4796-a2e0-e16a52bd4122 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:37.764868 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d084d1-78d8-4796-a2e0-e16a52bd4122 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:37.768476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d084d1-78d8-4796-a2e0-e16a52bd4122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:37.770463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f5a1e8b3-5ddd-4c10-9c32-e7a0c164e363 None None] MFA Rules not processed for user `eb2a8b66a4ef452ab3a84910841155aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:37.772610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f5a1e8b3-5ddd-4c10-9c32-e7a0c164e363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:37.816627 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a0d084d1-78d8-4796-a2e0-e16a52bd4122 None None] MFA Rules not processed for user `d1a1b2503bb84f2a9a9fb722c6d52cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:37.818422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a0d084d1-78d8-4796-a2e0-e16a52bd4122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:37.843896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:37.872445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1063/2116] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:37.888549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1e8b3-5ddd-4c10-9c32-e7a0c164e363 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:37.892061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f5a1e8b3-5ddd-4c10-9c32-e7a0c164e363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:37.907951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1054/2117] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:37.945909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:37.978080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1e8b3-5ddd-4c10-9c32-e7a0c164e363 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:37.983145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308bc0c-267a-4674-8cf9-54556be88100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:37.984469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308bc0c-267a-4674-8cf9-54556be88100 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:37.984469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308bc0c-267a-4674-8cf9-54556be88100 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:37.984469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308bc0c-267a-4674-8cf9-54556be88100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:37.986242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0308bc0c-267a-4674-8cf9-54556be88100 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:38.047397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308bc0c-267a-4674-8cf9-54556be88100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:38.048885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308bc0c-267a-4674-8cf9-54556be88100 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:38.058372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1064/2118] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:38.110807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d084d1-78d8-4796-a2e0-e16a52bd4122 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:38.112326 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a0d084d1-78d8-4796-a2e0-e16a52bd4122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:38.124557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d084d1-78d8-4796-a2e0-e16a52bd4122 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:38.129009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9998161-16b8-4a8e-b55f-5673a33598b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:38.133220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9998161-16b8-4a8e-b55f-5673a33598b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:38.133220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9998161-16b8-4a8e-b55f-5673a33598b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:38.133220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9998161-16b8-4a8e-b55f-5673a33598b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:38.141774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9998161-16b8-4a8e-b55f-5673a33598b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:38.143365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9998161-16b8-4a8e-b55f-5673a33598b4 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:38.154140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1055/2119] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:38.186104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308bc0c-267a-4674-8cf9-54556be88100 service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:38.278022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308bc0c-267a-4674-8cf9-54556be88100 service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:38.287017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:38.287017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:38.287017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:38.287017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] PATH_INFO: `/v3/users/b9f420d878074704a9707b30df462302` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:38.287422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f420d878074704a9707b30df462302)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:38.288933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:38.570190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f420d878074704a9707b30df462302'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:38.580083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f420d878074704a9707b30df462302'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:38.710431 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f420d878074704a9707b30df462302'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:38.711691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f420d878074704a9707b30df462302'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:38.753095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f420d878074704a9707b30df462302'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:38.760519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1065/2120] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:38 2022] DELETE /identity/v3/users/b9f420d878074704a9707b30df462302 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:38.765086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272cc87-9b25-458a-867a-177e13ab6fef None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:38.765410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272cc87-9b25-458a-867a-177e13ab6fef None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:38.765735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272cc87-9b25-458a-867a-177e13ab6fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:38.786472 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0272cc87-9b25-458a-867a-177e13ab6fef None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:38.787687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0272cc87-9b25-458a-867a-177e13ab6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:38.848462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:38.858415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1056/2121] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:38.869034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:38.870634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:38.977823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f231bcd-56de-4d69-b5c4-18297c5ba7cc admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:38.982337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f2cdb-24d7-4bd4-86c2-ca842aba4e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:38.983842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f2cdb-24d7-4bd4-86c2-ca842aba4e95 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:38.985007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f2cdb-24d7-4bd4-86c2-ca842aba4e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:38.985427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f2cdb-24d7-4bd4-86c2-ca842aba4e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:39.000963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f2cdb-24d7-4bd4-86c2-ca842aba4e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:39.002982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f2cdb-24d7-4bd4-86c2-ca842aba4e95 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:39.014643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1066/2122] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:39.481474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272cc87-9b25-458a-867a-177e13ab6fef None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:39.482944 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0272cc87-9b25-458a-867a-177e13ab6fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:39.494961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272cc87-9b25-458a-867a-177e13ab6fef None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:39.500076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:39.500738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:39.501071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:39.501351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] PATH_INFO: `/v3/projects/195d207cd1b547e8963ff59153c0c7dd` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:39.502567 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=195d207cd1b547e8963ff59153c0c7dd)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:39.504550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:39.645668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195d207cd1b547e8963ff59153c0c7dd', 'request_id': 'req-b8934e34-ca1a-4454-b163-e9bbee5e18a8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:39.651942 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195d207cd1b547e8963ff59153c0c7dd', 'request_id': 'req-b8934e34-ca1a-4454-b163-e9bbee5e18a8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:39.721486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195d207cd1b547e8963ff59153c0c7dd', 'request_id': 'req-b8934e34-ca1a-4454-b163-e9bbee5e18a8'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:39.734174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] The token cache is being invalidate because project 195d207cd1b547e8963ff59153c0c7dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:39.734393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:39.735057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:39.736721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1057/2123] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:39 2022] DELETE /identity/v3/projects/195d207cd1b547e8963ff59153c0c7dd => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:39.746803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f2cdb-24d7-4bd4-86c2-ca842aba4e95 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:39.770832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f2cdb-24d7-4bd4-86c2-ca842aba4e95 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:39.826742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:39.827384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:39.828364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:39.828649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] PATH_INFO: `/v3/users/d1a1b2503bb84f2a9a9fb722c6d52cbb` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:39.829838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a1b2503bb84f2a9a9fb722c6d52cbb)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:39.831753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:40.024994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a1b2503bb84f2a9a9fb722c6d52cbb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:40.032516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a1b2503bb84f2a9a9fb722c6d52cbb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:40.032681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:40.034107 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:40.066157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:40.085075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a1b2503bb84f2a9a9fb722c6d52cbb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:40.085697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a1b2503bb84f2a9a9fb722c6d52cbb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:40.091265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8934e34-ca1a-4454-b163-e9bbee5e18a8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:40.095056 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d1b43-ccd3-48af-b953-ac8a536d2268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:40.095675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d1b43-ccd3-48af-b953-ac8a536d2268 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:40.095979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d1b43-ccd3-48af-b953-ac8a536d2268 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:40.096252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d1b43-ccd3-48af-b953-ac8a536d2268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:40.098127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8e3d1b43-ccd3-48af-b953-ac8a536d2268 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:40.098792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a1b2503bb84f2a9a9fb722c6d52cbb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:40.105525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1067/2124] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:39 2022] DELETE /identity/v3/users/d1a1b2503bb84f2a9a9fb722c6d52cbb => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:40.165468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d1b43-ccd3-48af-b953-ac8a536d2268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:40.166924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d1b43-ccd3-48af-b953-ac8a536d2268 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:40.178409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1058/2125] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:40.555032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:40.556557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:40.584202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:40.635546 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c091906-e591-4285-b9bc-5609451a24d2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:40.642110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:40.642990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:40.643417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:40.643699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] PATH_INFO: `/v3/projects/6583d7b9e1e348d6a184e4fca5a7e2ae` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:40.645157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6583d7b9e1e348d6a184e4fca5a7e2ae)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:40.647058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:40.791056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6583d7b9e1e348d6a184e4fca5a7e2ae', 'request_id': 'req-71ae3b41-d739-40d9-9daa-5254d58b64d7'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:40.798568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6583d7b9e1e348d6a184e4fca5a7e2ae', 'request_id': 'req-71ae3b41-d739-40d9-9daa-5254d58b64d7'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:40.807110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6583d7b9e1e348d6a184e4fca5a7e2ae', 'request_id': 'req-71ae3b41-d739-40d9-9daa-5254d58b64d7'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:41.005607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] The token cache is being invalidate because project 6583d7b9e1e348d6a184e4fca5a7e2ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:41.005969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:41.006599 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71ae3b41-d739-40d9-9daa-5254d58b64d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:41.009689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1068/2126] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:40 2022] DELETE /identity/v3/projects/6583d7b9e1e348d6a184e4fca5a7e2ae => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:41.015842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85664811-2e31-442b-8aa1-db6e8f2b5f6a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.016350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85664811-2e31-442b-8aa1-db6e8f2b5f6a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.016429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85664811-2e31-442b-8aa1-db6e8f2b5f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.036491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-85664811-2e31-442b-8aa1-db6e8f2b5f6a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:41.037818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-85664811-2e31-442b-8aa1-db6e8f2b5f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:41.098699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1059/2127] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:41.106832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0c20b-a0b7-4771-bbea-771b03dba3d4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.107147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0c20b-a0b7-4771-bbea-771b03dba3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.107413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0c20b-a0b7-4771-bbea-771b03dba3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.127532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e7c0c20b-a0b7-4771-bbea-771b03dba3d4 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:41.129391 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e7c0c20b-a0b7-4771-bbea-771b03dba3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:41.136856 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1069/2128] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:41.141818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85664811-2e31-442b-8aa1-db6e8f2b5f6a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.151429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85664811-2e31-442b-8aa1-db6e8f2b5f6a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:41.156753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca830a-f7f5-478e-8fed-4ca1e7c07ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:41.157366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca830a-f7f5-478e-8fed-4ca1e7c07ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.158122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca830a-f7f5-478e-8fed-4ca1e7c07ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.158122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca830a-f7f5-478e-8fed-4ca1e7c07ca5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.158647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca830a-f7f5-478e-8fed-4ca1e7c07ca5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:41.158954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca830a-f7f5-478e-8fed-4ca1e7c07ca5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:41.160577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca830a-f7f5-478e-8fed-4ca1e7c07ca5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.172131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1060/2129] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:41.176480 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0c20b-a0b7-4771-bbea-771b03dba3d4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.185625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0c20b-a0b7-4771-bbea-771b03dba3d4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:41.193948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4eed4-48e6-443e-815f-18394e5707d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:41.194742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4eed4-48e6-443e-815f-18394e5707d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.195030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4eed4-48e6-443e-815f-18394e5707d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.195303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4eed4-48e6-443e-815f-18394e5707d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.196022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4eed4-48e6-443e-815f-18394e5707d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:41.198084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4eed4-48e6-443e-815f-18394e5707d4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.300825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1070/2130] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:41 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:41.305042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca830a-f7f5-478e-8fed-4ca1e7c07ca5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.315280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca830a-f7f5-478e-8fed-4ca1e7c07ca5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:41.371372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a0cce-f79f-4b6c-86d5-a387ea92630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:41.371955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a0cce-f79f-4b6c-86d5-a387ea92630d admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.372237 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a0cce-f79f-4b6c-86d5-a387ea92630d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.372509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a0cce-f79f-4b6c-86d5-a387ea92630d admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.373323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a0cce-f79f-4b6c-86d5-a387ea92630d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:41.375964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a0cce-f79f-4b6c-86d5-a387ea92630d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.387699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:41.474367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1061/2131] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:41 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:41.478760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4eed4-48e6-443e-815f-18394e5707d4 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.489367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4eed4-48e6-443e-815f-18394e5707d4 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:41.494059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e78f9-21fb-4bf9-81d7-93843b572e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:41.496674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e78f9-21fb-4bf9-81d7-93843b572e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.496674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e78f9-21fb-4bf9-81d7-93843b572e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.496674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e78f9-21fb-4bf9-81d7-93843b572e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.496674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e78f9-21fb-4bf9-81d7-93843b572e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:41.497685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e78f9-21fb-4bf9-81d7-93843b572e5e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.508196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1071/2132] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:41.512277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a0cce-f79f-4b6c-86d5-a387ea92630d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.525116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a0cce-f79f-4b6c-86d5-a387ea92630d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:41.530084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c0e23-b89f-4e00-9ae8-fda30113d546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:41.530688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c0e23-b89f-4e00-9ae8-fda30113d546 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.530968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c0e23-b89f-4e00-9ae8-fda30113d546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.531277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c0e23-b89f-4e00-9ae8-fda30113d546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.532062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c0e23-b89f-4e00-9ae8-fda30113d546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:41.533742 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c0e23-b89f-4e00-9ae8-fda30113d546 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.548371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1062/2133] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:41.554885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e78f9-21fb-4bf9-81d7-93843b572e5e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.560893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e78f9-21fb-4bf9-81d7-93843b572e5e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:41.567436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ae7ab-322a-40bb-80b7-ee2775636c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:41.568148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ae7ab-322a-40bb-80b7-ee2775636c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.568437 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ae7ab-322a-40bb-80b7-ee2775636c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.568721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ae7ab-322a-40bb-80b7-ee2775636c66 admin admin] PATH_INFO: `/v3/projects/d0348db9407b402ea2329b8e2a3b36af/users/a59dfcb4a35d4940a476953c95c6ad0a/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.589536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ae7ab-322a-40bb-80b7-ee2775636c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0348db9407b402ea2329b8e2a3b36af, user_id=a59dfcb4a35d4940a476953c95c6ad0a, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:41.595055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ae7ab-322a-40bb-80b7-ee2775636c66 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.668749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1072/2134] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:41 2022] PUT /identity/v3/projects/d0348db9407b402ea2329b8e2a3b36af/users/a59dfcb4a35d4940a476953c95c6ad0a/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:41.680443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503366b-e672-4a22-8385-4ac12f1422ef None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.680735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503366b-e672-4a22-8385-4ac12f1422ef None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.681013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503366b-e672-4a22-8385-4ac12f1422ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.709274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9503366b-e672-4a22-8385-4ac12f1422ef None None] MFA Rules not processed for user `a59dfcb4a35d4940a476953c95c6ad0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:41.710150 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9503366b-e672-4a22-8385-4ac12f1422ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:41.795639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1063/2135] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:41.800247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ae7ab-322a-40bb-80b7-ee2775636c66 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.860373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:41.877001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ae7ab-322a-40bb-80b7-ee2775636c66 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:41.882497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c3d43-726b-45c9-a83c-27b0d7bda7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:41.883221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c3d43-726b-45c9-a83c-27b0d7bda7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:41.883607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c3d43-726b-45c9-a83c-27b0d7bda7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:41.884164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c3d43-726b-45c9-a83c-27b0d7bda7d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:41.885182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c3d43-726b-45c9-a83c-27b0d7bda7d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:41.887119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c3d43-726b-45c9-a83c-27b0d7bda7d7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:41.984479 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1073/2136] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:41 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:41.986304 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503366b-e672-4a22-8385-4ac12f1422ef None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:41.998474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503366b-e672-4a22-8385-4ac12f1422ef None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:42.069232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c401d8d-c85b-4c46-bcf6-77fd41980995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:42.069870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c401d8d-c85b-4c46-bcf6-77fd41980995 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:42.070155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c401d8d-c85b-4c46-bcf6-77fd41980995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:42.070457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c401d8d-c85b-4c46-bcf6-77fd41980995 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:42.071123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c401d8d-c85b-4c46-bcf6-77fd41980995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:42.073133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c401d8d-c85b-4c46-bcf6-77fd41980995 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:42.216241 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1064/2137] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:41 2022] POST /identity/v3/users => generated 298 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:42.217421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c3d43-726b-45c9-a83c-27b0d7bda7d7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:42.228334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c3d43-726b-45c9-a83c-27b0d7bda7d7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:42.237001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6187f1-9907-4381-85fb-d915f6b42001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:42.237672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6187f1-9907-4381-85fb-d915f6b42001 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:42.237962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6187f1-9907-4381-85fb-d915f6b42001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:42.238263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6187f1-9907-4381-85fb-d915f6b42001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:42.239823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6187f1-9907-4381-85fb-d915f6b42001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:42.242570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6187f1-9907-4381-85fb-d915f6b42001 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:42.258642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1074/2138] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:42.259119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c401d8d-c85b-4c46-bcf6-77fd41980995 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:42.269179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c401d8d-c85b-4c46-bcf6-77fd41980995 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:42.273808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550175b-bb87-4513-b1fd-7a7c0f1a064b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:42.274478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550175b-bb87-4513-b1fd-7a7c0f1a064b admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:42.274766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550175b-bb87-4513-b1fd-7a7c0f1a064b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:42.275047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550175b-bb87-4513-b1fd-7a7c0f1a064b admin admin] PATH_INFO: `/v3/projects/cc8a40fb941842c7bbcc3cd6aa98e352/users/b58a2748fe724878a9660bca29666335/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:42.291313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550175b-bb87-4513-b1fd-7a7c0f1a064b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc8a40fb941842c7bbcc3cd6aa98e352, user_id=b58a2748fe724878a9660bca29666335, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:42.294014 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550175b-bb87-4513-b1fd-7a7c0f1a064b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:42.374402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1065/2139] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:42 2022] PUT /identity/v3/projects/cc8a40fb941842c7bbcc3cd6aa98e352/users/b58a2748fe724878a9660bca29666335/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:42.378768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6187f1-9907-4381-85fb-d915f6b42001 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:42.389610 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6187f1-9907-4381-85fb-d915f6b42001 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:42.470614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc5bdc-e342-40eb-b1bb-2b465dbf1671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:42.471244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc5bdc-e342-40eb-b1bb-2b465dbf1671 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:42.471527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc5bdc-e342-40eb-b1bb-2b465dbf1671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:42.471835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc5bdc-e342-40eb-b1bb-2b465dbf1671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:42.472542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc5bdc-e342-40eb-b1bb-2b465dbf1671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:42.474146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc5bdc-e342-40eb-b1bb-2b465dbf1671 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:42.487830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1075/2140] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:42.490096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550175b-bb87-4513-b1fd-7a7c0f1a064b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:42.501350 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550175b-bb87-4513-b1fd-7a7c0f1a064b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:42.511035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646a39a-f45b-409f-8a7d-5673e236a1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:42.511905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646a39a-f45b-409f-8a7d-5673e236a1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:42.512193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646a39a-f45b-409f-8a7d-5673e236a1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:42.512557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646a39a-f45b-409f-8a7d-5673e236a1a9 admin admin] PATH_INFO: `/v3/projects/cc8a40fb941842c7bbcc3cd6aa98e352/users/b58a2748fe724878a9660bca29666335/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:42.519545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646a39a-f45b-409f-8a7d-5673e236a1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc8a40fb941842c7bbcc3cd6aa98e352, user_id=b58a2748fe724878a9660bca29666335, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:42.524975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646a39a-f45b-409f-8a7d-5673e236a1a9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:42.547439 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1066/2141] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:42 2022] PUT /identity/v3/projects/cc8a40fb941842c7bbcc3cd6aa98e352/users/b58a2748fe724878a9660bca29666335/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:42.565945 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3bebd-ca2f-4062-ba6d-905597130336 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:42.566642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3bebd-ca2f-4062-ba6d-905597130336 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:42.570812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3bebd-ca2f-4062-ba6d-905597130336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:42.638009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-eaf3bebd-ca2f-4062-ba6d-905597130336 None None] MFA Rules not processed for user `b58a2748fe724878a9660bca29666335`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:42.638009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-eaf3bebd-ca2f-4062-ba6d-905597130336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:42.826043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1076/2142] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:42.908249 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646a39a-f45b-409f-8a7d-5673e236a1a9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:42.909935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9646a39a-f45b-409f-8a7d-5673e236a1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:42.941787 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:42.969987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646a39a-f45b-409f-8a7d-5673e236a1a9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:42.974613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1c40a-8663-4fe6-b7de-d4c7e0384f77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:42.974613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c40a-8663-4fe6-b7de-d4c7e0384f77 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:42.974613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c40a-8663-4fe6-b7de-d4c7e0384f77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:42.974871 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c40a-8663-4fe6-b7de-d4c7e0384f77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:42.984735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1c40a-8663-4fe6-b7de-d4c7e0384f77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:42.986256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1c40a-8663-4fe6-b7de-d4c7e0384f77 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:42.995845 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1067/2143] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:43.391478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3bebd-ca2f-4062-ba6d-905597130336 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:43.392958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-eaf3bebd-ca2f-4062-ba6d-905597130336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:43.404586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3bebd-ca2f-4062-ba6d-905597130336 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:43.408744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f94c0-aa25-48c9-bceb-26fe09dba6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:43.409374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f94c0-aa25-48c9-bceb-26fe09dba6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:43.409648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f94c0-aa25-48c9-bceb-26fe09dba6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:43.409919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f94c0-aa25-48c9-bceb-26fe09dba6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:43.448935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:43.465990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f94c0-aa25-48c9-bceb-26fe09dba6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:43.467476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f94c0-aa25-48c9-bceb-26fe09dba6f4 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:43.477485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1077/2144] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:44.467328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1c40a-8663-4fe6-b7de-d4c7e0384f77 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:44.556563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1c40a-8663-4fe6-b7de-d4c7e0384f77 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:44.562804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:44.563527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:44.563849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:44.564124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] PATH_INFO: `/v3/users/a59dfcb4a35d4940a476953c95c6ad0a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:44.565578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a59dfcb4a35d4940a476953c95c6ad0a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:44.569684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:44.804063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a59dfcb4a35d4940a476953c95c6ad0a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:44.812701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a59dfcb4a35d4940a476953c95c6ad0a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:44.902400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a59dfcb4a35d4940a476953c95c6ad0a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:44.903957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a59dfcb4a35d4940a476953c95c6ad0a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:44.906819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a59dfcb4a35d4940a476953c95c6ad0a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:44.929274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1068/2145] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:44 2022] DELETE /identity/v3/users/a59dfcb4a35d4940a476953c95c6ad0a => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:44.933902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75665-8418-4e57-b731-e99350168a31 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:44.934369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75665-8418-4e57-b731-e99350168a31 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:44.934476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75665-8418-4e57-b731-e99350168a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:44.957032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-62d75665-8418-4e57-b731-e99350168a31 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:44.958294 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-62d75665-8418-4e57-b731-e99350168a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:45.038255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1078/2146] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:45.048582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:45.050179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:45.112756 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:45.116663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80677221-0fe7-4b25-89f8-fed932cfbfdd admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:45.120412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cc84d-a8f1-446b-ace1-122e570965b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:45.121053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cc84d-a8f1-446b-ace1-122e570965b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:45.121337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cc84d-a8f1-446b-ace1-122e570965b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:45.121614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cc84d-a8f1-446b-ace1-122e570965b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:45.131341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cc84d-a8f1-446b-ace1-122e570965b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:45.132827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cc84d-a8f1-446b-ace1-122e570965b5 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:45.142830 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1069/2147] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:45.729924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d75665-8418-4e57-b731-e99350168a31 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:45.731772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-62d75665-8418-4e57-b731-e99350168a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:45.763966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d75665-8418-4e57-b731-e99350168a31 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:45.776198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:45.776861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:45.777145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:45.777417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] PATH_INFO: `/v3/projects/d0348db9407b402ea2329b8e2a3b36af` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:45.778582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0348db9407b402ea2329b8e2a3b36af)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:45.780683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:45.966962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0348db9407b402ea2329b8e2a3b36af', 'request_id': 'req-0d90a610-4caa-4e39-a4bc-7e09baf02232'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:45.974879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0348db9407b402ea2329b8e2a3b36af', 'request_id': 'req-0d90a610-4caa-4e39-a4bc-7e09baf02232'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:45.981898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0348db9407b402ea2329b8e2a3b36af', 'request_id': 'req-0d90a610-4caa-4e39-a4bc-7e09baf02232'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:46.090612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] The token cache is being invalidate because project d0348db9407b402ea2329b8e2a3b36af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:46.090612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:46.090612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:46.092276 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1079/2148] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:45 2022] DELETE /identity/v3/projects/d0348db9407b402ea2329b8e2a3b36af => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:46.096189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cc84d-a8f1-446b-ace1-122e570965b5 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:46.106659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cc84d-a8f1-446b-ace1-122e570965b5 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:46.165816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:46.166464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:46.166747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:46.167022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] PATH_INFO: `/v3/users/b58a2748fe724878a9660bca29666335` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:46.168220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b58a2748fe724878a9660bca29666335)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:46.170212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:46.358704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b58a2748fe724878a9660bca29666335'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:46.365930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b58a2748fe724878a9660bca29666335'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:46.433542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b58a2748fe724878a9660bca29666335'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:46.434160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b58a2748fe724878a9660bca29666335'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:46.434770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2ce598ed-32b8-4af1-98a4-9a5293017435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b58a2748fe724878a9660bca29666335'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:46.452265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1070/2149] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:46 2022] DELETE /identity/v3/users/b58a2748fe724878a9660bca29666335 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:47.124455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:47.127229 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:47.145173 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90a610-4caa-4e39-a4bc-7e09baf02232 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:47.203554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:47.204232 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:47.204583 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:47.204862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] PATH_INFO: `/v3/projects/cc8a40fb941842c7bbcc3cd6aa98e352` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:47.206076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc8a40fb941842c7bbcc3cd6aa98e352)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:47.208005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:47.214329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:47.326608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc8a40fb941842c7bbcc3cd6aa98e352', 'request_id': 'req-d4c118d3-1fd4-410c-b380-3f49581aad59'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:47.333202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc8a40fb941842c7bbcc3cd6aa98e352', 'request_id': 'req-d4c118d3-1fd4-410c-b380-3f49581aad59'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:47.339159 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc8a40fb941842c7bbcc3cd6aa98e352', 'request_id': 'req-d4c118d3-1fd4-410c-b380-3f49581aad59'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:47.421342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] The token cache is being invalidate because project cc8a40fb941842c7bbcc3cd6aa98e352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:47.421789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:47.422505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4c118d3-1fd4-410c-b380-3f49581aad59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:47.424079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1080/2150] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:47 2022] DELETE /identity/v3/projects/cc8a40fb941842c7bbcc3cd6aa98e352 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:47.429014 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625801d-2096-43d5-a3e2-df1f39e624bb None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:47.429456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625801d-2096-43d5-a3e2-df1f39e624bb None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:47.429571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625801d-2096-43d5-a3e2-df1f39e624bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:47.449130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8625801d-2096-43d5-a3e2-df1f39e624bb None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:47.450379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8625801d-2096-43d5-a3e2-df1f39e624bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:47.513969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1071/2151] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:47.522268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd651f-3cab-4f53-b62f-03ecc08d2413 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:47.522554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd651f-3cab-4f53-b62f-03ecc08d2413 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:47.522838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd651f-3cab-4f53-b62f-03ecc08d2413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:47.543442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b0fd651f-3cab-4f53-b62f-03ecc08d2413 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:47.544652 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b0fd651f-3cab-4f53-b62f-03ecc08d2413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:47.552673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1081/2152] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:47.556816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625801d-2096-43d5-a3e2-df1f39e624bb None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:47.566925 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625801d-2096-43d5-a3e2-df1f39e624bb None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:47.571475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222655-89b1-45c6-9329-a6606842997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:47.573945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222655-89b1-45c6-9329-a6606842997d admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:47.574230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222655-89b1-45c6-9329-a6606842997d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:47.574508 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222655-89b1-45c6-9329-a6606842997d admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:47.575207 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa222655-89b1-45c6-9329-a6606842997d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:47.575522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa222655-89b1-45c6-9329-a6606842997d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:47.577246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa222655-89b1-45c6-9329-a6606842997d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:47.584742 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:47.589701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1072/2153] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:47.594131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd651f-3cab-4f53-b62f-03ecc08d2413 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:47.603270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd651f-3cab-4f53-b62f-03ecc08d2413 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:47.608176 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdba14-d47f-4e04-b2b3-da37af653b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:47.608776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdba14-d47f-4e04-b2b3-da37af653b5a admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:47.609092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdba14-d47f-4e04-b2b3-da37af653b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:47.609368 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdba14-d47f-4e04-b2b3-da37af653b5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:47.610024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdba14-d47f-4e04-b2b3-da37af653b5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:47.611925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdba14-d47f-4e04-b2b3-da37af653b5a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:47.686212 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1082/2154] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:47 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:47.689192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222655-89b1-45c6-9329-a6606842997d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:47.699043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222655-89b1-45c6-9329-a6606842997d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:47.754986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e718cf9-5f74-4556-ad99-7b80d20845f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:47.755639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e718cf9-5f74-4556-ad99-7b80d20845f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:47.755952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e718cf9-5f74-4556-ad99-7b80d20845f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:47.756228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e718cf9-5f74-4556-ad99-7b80d20845f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:47.756894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e718cf9-5f74-4556-ad99-7b80d20845f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:47.758816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e718cf9-5f74-4556-ad99-7b80d20845f7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:47.848052 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1073/2155] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:47 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:47.852034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdba14-d47f-4e04-b2b3-da37af653b5a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:47.862657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdba14-d47f-4e04-b2b3-da37af653b5a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:47.867505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b4c2b-a005-4ca6-968e-a4eae68c1ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:47.869464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b4c2b-a005-4ca6-968e-a4eae68c1ddd admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:47.869464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b4c2b-a005-4ca6-968e-a4eae68c1ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:47.869464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b4c2b-a005-4ca6-968e-a4eae68c1ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:47.869464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b4c2b-a005-4ca6-968e-a4eae68c1ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:47.870925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b4c2b-a005-4ca6-968e-a4eae68c1ddd admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:47.880771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1083/2156] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:47.884420 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e718cf9-5f74-4556-ad99-7b80d20845f7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:47.894904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e718cf9-5f74-4556-ad99-7b80d20845f7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:47.899622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccaa4d6-3015-4d31-91a3-b9560f44b8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:47.900265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaa4d6-3015-4d31-91a3-b9560f44b8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:47.900552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaa4d6-3015-4d31-91a3-b9560f44b8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:47.900828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaa4d6-3015-4d31-91a3-b9560f44b8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:47.901518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaa4d6-3015-4d31-91a3-b9560f44b8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:47.903112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaa4d6-3015-4d31-91a3-b9560f44b8b6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:47.914191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1074/2157] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:47.918389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b4c2b-a005-4ca6-968e-a4eae68c1ddd admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:47.928183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b4c2b-a005-4ca6-968e-a4eae68c1ddd admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:47.932812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29abc9f-8c1d-4da5-9d02-8692ab368c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:47.933463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29abc9f-8c1d-4da5-9d02-8692ab368c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:47.933749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29abc9f-8c1d-4da5-9d02-8692ab368c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:47.934028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29abc9f-8c1d-4da5-9d02-8692ab368c78 admin admin] PATH_INFO: `/v3/projects/0cbb6f2bc24f46749e888bbf98906330/users/4482913e75f1480ab35e723124aa44ac/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:47.948078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29abc9f-8c1d-4da5-9d02-8692ab368c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cbb6f2bc24f46749e888bbf98906330, user_id=4482913e75f1480ab35e723124aa44ac, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:47.950705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29abc9f-8c1d-4da5-9d02-8692ab368c78 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:48.036880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1084/2158] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:47 2022] PUT /identity/v3/projects/0cbb6f2bc24f46749e888bbf98906330/users/4482913e75f1480ab35e723124aa44ac/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:48.048148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db0745-d0bc-4885-a675-33080de25773 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.048437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db0745-d0bc-4885-a675-33080de25773 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.048739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db0745-d0bc-4885-a675-33080de25773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.075542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c0db0745-d0bc-4885-a675-33080de25773 None None] MFA Rules not processed for user `4482913e75f1480ab35e723124aa44ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:48.076769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c0db0745-d0bc-4885-a675-33080de25773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:48.144138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1075/2159] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:48.148714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29abc9f-8c1d-4da5-9d02-8692ab368c78 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:48.244188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29abc9f-8c1d-4da5-9d02-8692ab368c78 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:48.249415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9257d3-6581-428a-98c9-537a5c766a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:48.250083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9257d3-6581-428a-98c9-537a5c766a85 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.250397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9257d3-6581-428a-98c9-537a5c766a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.250679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9257d3-6581-428a-98c9-537a5c766a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.251390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9257d3-6581-428a-98c9-537a5c766a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:48.253378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9257d3-6581-428a-98c9-537a5c766a85 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:48.337144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1085/2160] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:48 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:48.340368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db0745-d0bc-4885-a675-33080de25773 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:48.355773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db0745-d0bc-4885-a675-33080de25773 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:48.411460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238d578-fd2f-4e2f-b9d8-852da5cb3fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:48.412066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238d578-fd2f-4e2f-b9d8-852da5cb3fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.412352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238d578-fd2f-4e2f-b9d8-852da5cb3fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.412629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238d578-fd2f-4e2f-b9d8-852da5cb3fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.413290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238d578-fd2f-4e2f-b9d8-852da5cb3fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:48.415230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238d578-fd2f-4e2f-b9d8-852da5cb3fe8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:48.426370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:48.544400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1076/2161] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:48 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:48.548392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9257d3-6581-428a-98c9-537a5c766a85 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:48.559632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9257d3-6581-428a-98c9-537a5c766a85 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:48.564554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10ff05-6c33-46be-9211-ef4e5d5dbf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:48.565183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10ff05-6c33-46be-9211-ef4e5d5dbf8d admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.565465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10ff05-6c33-46be-9211-ef4e5d5dbf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.565740 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10ff05-6c33-46be-9211-ef4e5d5dbf8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.566447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10ff05-6c33-46be-9211-ef4e5d5dbf8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:48.568154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10ff05-6c33-46be-9211-ef4e5d5dbf8d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:48.582787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1086/2162] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:48.584052 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238d578-fd2f-4e2f-b9d8-852da5cb3fe8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:48.595157 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238d578-fd2f-4e2f-b9d8-852da5cb3fe8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:48.604527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35681890-adb3-4426-bc6d-b213e74f83b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:48.605165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35681890-adb3-4426-bc6d-b213e74f83b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.605717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35681890-adb3-4426-bc6d-b213e74f83b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.605999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35681890-adb3-4426-bc6d-b213e74f83b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.606704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35681890-adb3-4426-bc6d-b213e74f83b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:48.610506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35681890-adb3-4426-bc6d-b213e74f83b2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:48.624601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1077/2163] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:48.628804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10ff05-6c33-46be-9211-ef4e5d5dbf8d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:48.638271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:48.643197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10ff05-6c33-46be-9211-ef4e5d5dbf8d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:48.648540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1b3ec-45f3-4921-8e22-26fee3709305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:48.649220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1b3ec-45f3-4921-8e22-26fee3709305 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.650594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1b3ec-45f3-4921-8e22-26fee3709305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.650594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1b3ec-45f3-4921-8e22-26fee3709305 admin admin] PATH_INFO: `/v3/projects/7f30ceea6ec742c08afcf636c51a7e65/users/d04069adb90e42798f6d3a79c1dc28b2/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.665989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1b3ec-45f3-4921-8e22-26fee3709305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f30ceea6ec742c08afcf636c51a7e65, user_id=d04069adb90e42798f6d3a79c1dc28b2, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:48.668733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e1b3ec-45f3-4921-8e22-26fee3709305 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:48.809017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1087/2164] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:48 2022] PUT /identity/v3/projects/7f30ceea6ec742c08afcf636c51a7e65/users/d04069adb90e42798f6d3a79c1dc28b2/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:48.819716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879902a-c601-4f95-bb82-36c2f336fcb0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.820055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879902a-c601-4f95-bb82-36c2f336fcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.820344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879902a-c601-4f95-bb82-36c2f336fcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.849953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b879902a-c601-4f95-bb82-36c2f336fcb0 None None] MFA Rules not processed for user `d04069adb90e42798f6d3a79c1dc28b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:48.851218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b879902a-c601-4f95-bb82-36c2f336fcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:48.918068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1078/2165] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:48.922623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1b3ec-45f3-4921-8e22-26fee3709305 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:48.985058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1b3ec-45f3-4921-8e22-26fee3709305 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:48.989765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb250a-129a-43e2-8513-961b87b59338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:48.990380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb250a-129a-43e2-8513-961b87b59338 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:48.990657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb250a-129a-43e2-8513-961b87b59338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:48.990931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb250a-129a-43e2-8513-961b87b59338 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:48.991600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb250a-129a-43e2-8513-961b87b59338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:48.993507 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb250a-129a-43e2-8513-961b87b59338 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.094357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1088/2166] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:48 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:49.098620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879902a-c601-4f95-bb82-36c2f336fcb0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.110110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879902a-c601-4f95-bb82-36c2f336fcb0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.167647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe946af-6ebf-4b1d-bd04-5ce870e1afcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.168466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe946af-6ebf-4b1d-bd04-5ce870e1afcf admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.168690 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe946af-6ebf-4b1d-bd04-5ce870e1afcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.168968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe946af-6ebf-4b1d-bd04-5ce870e1afcf admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.169864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe946af-6ebf-4b1d-bd04-5ce870e1afcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.172474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe946af-6ebf-4b1d-bd04-5ce870e1afcf admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.255222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1079/2167] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:49 2022] POST /identity/v3/users => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:49.261415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb250a-129a-43e2-8513-961b87b59338 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.278862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb250a-129a-43e2-8513-961b87b59338 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.288295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02432678-1f42-4b0b-9b5c-66959ab35136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.289214 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02432678-1f42-4b0b-9b5c-66959ab35136 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.289669 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02432678-1f42-4b0b-9b5c-66959ab35136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.290183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02432678-1f42-4b0b-9b5c-66959ab35136 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.292004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02432678-1f42-4b0b-9b5c-66959ab35136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.294181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02432678-1f42-4b0b-9b5c-66959ab35136 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.311025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1089/2168] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:49.312373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe946af-6ebf-4b1d-bd04-5ce870e1afcf admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.336654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe946af-6ebf-4b1d-bd04-5ce870e1afcf admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.342051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b0c5e-ec74-4646-9ff2-323011775078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.342839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b0c5e-ec74-4646-9ff2-323011775078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.343274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b0c5e-ec74-4646-9ff2-323011775078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.344279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b0c5e-ec74-4646-9ff2-323011775078 admin admin] PATH_INFO: `/v3/projects/34e9f79920644d838392fcffe4854035/users/a78b3ae46f8d4c549cba9d2a17204fa9/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.371628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b0c5e-ec74-4646-9ff2-323011775078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e9f79920644d838392fcffe4854035, user_id=a78b3ae46f8d4c549cba9d2a17204fa9, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.376116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b0c5e-ec74-4646-9ff2-323011775078 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.506268 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1080/2169] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:49 2022] PUT /identity/v3/projects/34e9f79920644d838392fcffe4854035/users/a78b3ae46f8d4c549cba9d2a17204fa9/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:49.510306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02432678-1f42-4b0b-9b5c-66959ab35136 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.521463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02432678-1f42-4b0b-9b5c-66959ab35136 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.598917 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7df72-04d0-49d1-9172-d23fc42e8b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.599807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7df72-04d0-49d1-9172-d23fc42e8b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.601177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7df72-04d0-49d1-9172-d23fc42e8b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.601714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7df72-04d0-49d1-9172-d23fc42e8b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.602947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7df72-04d0-49d1-9172-d23fc42e8b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.607352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7df72-04d0-49d1-9172-d23fc42e8b5b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.621137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1090/2170] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:49.623320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b0c5e-ec74-4646-9ff2-323011775078 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.636398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b0c5e-ec74-4646-9ff2-323011775078 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:49.641900 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e5cb1-13f7-498e-a717-2d250585397d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:49.643513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e5cb1-13f7-498e-a717-2d250585397d admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.643823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e5cb1-13f7-498e-a717-2d250585397d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.644102 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e5cb1-13f7-498e-a717-2d250585397d admin admin] PATH_INFO: `/v3/projects/34e9f79920644d838392fcffe4854035/users/a78b3ae46f8d4c549cba9d2a17204fa9/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.646450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e5cb1-13f7-498e-a717-2d250585397d admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e9f79920644d838392fcffe4854035, user_id=a78b3ae46f8d4c549cba9d2a17204fa9, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:49.649354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e5cb1-13f7-498e-a717-2d250585397d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:49.664809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1081/2171] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:49 2022] PUT /identity/v3/projects/34e9f79920644d838392fcffe4854035/users/a78b3ae46f8d4c549cba9d2a17204fa9/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:49.680121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6faefc-8fe4-477d-a1c4-ecd10465d998 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:49.680121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6faefc-8fe4-477d-a1c4-ecd10465d998 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:49.680121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6faefc-8fe4-477d-a1c4-ecd10465d998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:49.720801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1d6faefc-8fe4-477d-a1c4-ecd10465d998 None None] MFA Rules not processed for user `a78b3ae46f8d4c549cba9d2a17204fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:49.722575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1d6faefc-8fe4-477d-a1c4-ecd10465d998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:49.775039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:49.813920 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1091/2172] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:49.950887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e5cb1-13f7-498e-a717-2d250585397d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:49.954621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-305e5cb1-13f7-498e-a717-2d250585397d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:49.995070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:50.035294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e5cb1-13f7-498e-a717-2d250585397d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:50.040545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bed59c-ebb4-4ac1-8ad2-c9fb5b4f95f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:50.041314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bed59c-ebb4-4ac1-8ad2-c9fb5b4f95f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:50.041685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bed59c-ebb4-4ac1-8ad2-c9fb5b4f95f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:50.042340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bed59c-ebb4-4ac1-8ad2-c9fb5b4f95f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:50.103763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bed59c-ebb4-4ac1-8ad2-c9fb5b4f95f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:50.105299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bed59c-ebb4-4ac1-8ad2-c9fb5b4f95f4 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:50.115146 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1082/2173] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:52.307870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6faefc-8fe4-477d-a1c4-ecd10465d998 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:52.309281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1d6faefc-8fe4-477d-a1c4-ecd10465d998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:52.322001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6faefc-8fe4-477d-a1c4-ecd10465d998 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:52.325737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8a152-9688-4b8f-85b0-5663f77129dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:52.326600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a152-9688-4b8f-85b0-5663f77129dd service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:52.326880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a152-9688-4b8f-85b0-5663f77129dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:52.327151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a152-9688-4b8f-85b0-5663f77129dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:52.402116 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8a152-9688-4b8f-85b0-5663f77129dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:52.403700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8a152-9688-4b8f-85b0-5663f77129dd service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:52.420051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1092/2174] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:52.956771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bed59c-ebb4-4ac1-8ad2-c9fb5b4f95f4 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:52.958341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-31bed59c-ebb4-4ac1-8ad2-c9fb5b4f95f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:53.054968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bed59c-ebb4-4ac1-8ad2-c9fb5b4f95f4 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:53.074027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:53.074027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.074027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.074027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] PATH_INFO: `/v3/users/c8fd5f27c0a742239e1e5851c92dd30f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.076585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8fd5f27c0a742239e1e5851c92dd30f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:53.080842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:53.332749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8fd5f27c0a742239e1e5851c92dd30f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:53.361238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8fd5f27c0a742239e1e5851c92dd30f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:53.447912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8fd5f27c0a742239e1e5851c92dd30f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:53.448550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8fd5f27c0a742239e1e5851c92dd30f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:53.450996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8fd5f27c0a742239e1e5851c92dd30f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:53.467015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:53.475585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1083/2175] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:52 2022] DELETE /identity/v3/users/c8fd5f27c0a742239e1e5851c92dd30f => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:53.477110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1c2c-2263-4b4b-a849-10c4ee1be77d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.477566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1c2c-2263-4b4b-a849-10c4ee1be77d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.477648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1c2c-2263-4b4b-a849-10c4ee1be77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.502426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-df3b1c2c-2263-4b4b-a849-10c4ee1be77d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:53.503812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-df3b1c2c-2263-4b4b-a849-10c4ee1be77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:53.594722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1093/2176] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:53.615668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:53.617311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:53.636524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b9c4c-d6f2-497e-aab9-0048cbe0407a None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.639131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b9c4c-d6f2-497e-aab9-0048cbe0407a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.639131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b9c4c-d6f2-497e-aab9-0048cbe0407a None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.639131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1094/2177] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:23:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 01:23:53.665211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2ab8e-967c-4608-baac-75bbaf9a43dd None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.665978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2ab8e-967c-4608-baac-75bbaf9a43dd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.666265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2ab8e-967c-4608-baac-75bbaf9a43dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.684322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa0e7d-8fae-437f-8565-69a108e30ce3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:53.687927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc7432-3ebc-4b1d-9b9a-d2dea3657219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:53.688139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-80b2ab8e-967c-4608-baac-75bbaf9a43dd None None] MFA Rules not processed for user `8361f2f9c70b4a3288a187cb2d97d415`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:53.688569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc7432-3ebc-4b1d-9b9a-d2dea3657219 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.688883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc7432-3ebc-4b1d-9b9a-d2dea3657219 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.689152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc7432-3ebc-4b1d-9b9a-d2dea3657219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.689703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-80b2ab8e-967c-4608-baac-75bbaf9a43dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:53.699799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc7432-3ebc-4b1d-9b9a-d2dea3657219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:53.701253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc7432-3ebc-4b1d-9b9a-d2dea3657219 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:53.711964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1084/2178] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:53.763118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1095/2179] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:23:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:53.775443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc7432-3ebc-4b1d-9b9a-d2dea3657219 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:53.776730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-98bc7432-3ebc-4b1d-9b9a-d2dea3657219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:53.792471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc7432-3ebc-4b1d-9b9a-d2dea3657219 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:53.801369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d8aa3-a5cc-402f-affc-dbf2910bc800 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:53.802066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d8aa3-a5cc-402f-affc-dbf2910bc800 service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.813852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d8aa3-a5cc-402f-affc-dbf2910bc800 service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.813852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d8aa3-a5cc-402f-affc-dbf2910bc800 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.814358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2ab8e-967c-4608-baac-75bbaf9a43dd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:53.814358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-80b2ab8e-967c-4608-baac-75bbaf9a43dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:53.817394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d8aa3-a5cc-402f-affc-dbf2910bc800 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:53.820422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d8aa3-a5cc-402f-affc-dbf2910bc800 service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:53.838895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2ab8e-967c-4608-baac-75bbaf9a43dd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:53.841504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1085/2180] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:53.844425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:53.845067 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:53.845349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:53.845624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] PATH_INFO: `/v3/users/90f54ffc10a447d0a873f3e7c6ab2647` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:53.846888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f54ffc10a447d0a873f3e7c6ab2647)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:53.848878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:53.854190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:54.108121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f54ffc10a447d0a873f3e7c6ab2647'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.119115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f54ffc10a447d0a873f3e7c6ab2647'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.194206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f54ffc10a447d0a873f3e7c6ab2647'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.194848 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f54ffc10a447d0a873f3e7c6ab2647'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.202132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d8aa3-a5cc-402f-affc-dbf2910bc800 service placement] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:54.204424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ae2d8aa3-a5cc-402f-affc-dbf2910bc800 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:54.221640 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d8aa3-a5cc-402f-affc-dbf2910bc800 service placement] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:54.226211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:54.226894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.227188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.227449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] PATH_INFO: `/v3/projects/64a2eddfc6494005aeda51bec7fb256b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.228729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a2eddfc6494005aeda51bec7fb256b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:54.231011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:54.240213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f54ffc10a447d0a873f3e7c6ab2647'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.247654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1096/2181] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:53 2022] DELETE /identity/v3/users/90f54ffc10a447d0a873f3e7c6ab2647 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:54.368530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a2eddfc6494005aeda51bec7fb256b', 'request_id': 'req-7c16df0a-7b84-4d0e-8756-629cf8096a8a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.382245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a2eddfc6494005aeda51bec7fb256b', 'request_id': 'req-7c16df0a-7b84-4d0e-8756-629cf8096a8a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.460377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a2eddfc6494005aeda51bec7fb256b', 'request_id': 'req-7c16df0a-7b84-4d0e-8756-629cf8096a8a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.476698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:54.478252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:54.492489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] The token cache is being invalidate because project 64a2eddfc6494005aeda51bec7fb256b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:54.492797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.494751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7c16df0a-7b84-4d0e-8756-629cf8096a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.506465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1086/2182] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:54 2022] DELETE /identity/v3/projects/64a2eddfc6494005aeda51bec7fb256b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:54.508107 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415c68c-529c-4239-a7ed-1ef05f570f84 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.508401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415c68c-529c-4239-a7ed-1ef05f570f84 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.508685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415c68c-529c-4239-a7ed-1ef05f570f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.551621 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50980d83-17ce-4263-af41-b350270cde05 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:54.561182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8415c68c-529c-4239-a7ed-1ef05f570f84 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:54.563009 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8415c68c-529c-4239-a7ed-1ef05f570f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:54.641852 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238b6f6-0222-43c9-a4cf-e0bc91ab6057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:54.642820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238b6f6-0222-43c9-a4cf-e0bc91ab6057 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.643288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238b6f6-0222-43c9-a4cf-e0bc91ab6057 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.643687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238b6f6-0222-43c9-a4cf-e0bc91ab6057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.645756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0238b6f6-0222-43c9-a4cf-e0bc91ab6057 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:54.652366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1087/2183] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:54.653730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415c68c-529c-4239-a7ed-1ef05f570f84 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:54.655316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8415c68c-529c-4239-a7ed-1ef05f570f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:54.674269 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415c68c-529c-4239-a7ed-1ef05f570f84 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:54.675351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:54.676049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.676334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.676666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] PATH_INFO: `/v3/projects/ced29d168e064480a2d3619a7d8fca88` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.677886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ced29d168e064480a2d3619a7d8fca88)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:54.682048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:54.690699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:54.715821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:54.731217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238b6f6-0222-43c9-a4cf-e0bc91ab6057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:54.733137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238b6f6-0222-43c9-a4cf-e0bc91ab6057 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:54.743379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1097/2184] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:54.744661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41eb90-faa9-4e9e-81c5-25a5ab16a3b8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.755015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41eb90-faa9-4e9e-81c5-25a5ab16a3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.755015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41eb90-faa9-4e9e-81c5-25a5ab16a3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.766543 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6b41eb90-faa9-4e9e-81c5-25a5ab16a3b8 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:54.767770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6b41eb90-faa9-4e9e-81c5-25a5ab16a3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:54.775899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1098/2185] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:54.780816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41eb90-faa9-4e9e-81c5-25a5ab16a3b8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:54.790395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41eb90-faa9-4e9e-81c5-25a5ab16a3b8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:54.795955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099627c2-6d87-483c-a8a4-335edd5f48ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:54.797296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099627c2-6d87-483c-a8a4-335edd5f48ca admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.797557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099627c2-6d87-483c-a8a4-335edd5f48ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.797745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099627c2-6d87-483c-a8a4-335edd5f48ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.798727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099627c2-6d87-483c-a8a4-335edd5f48ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:54.799037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099627c2-6d87-483c-a8a4-335edd5f48ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:54.801470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099627c2-6d87-483c-a8a4-335edd5f48ca admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:54.819041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1099/2186] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:54.823256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099627c2-6d87-483c-a8a4-335edd5f48ca admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:54.830246 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ced29d168e064480a2d3619a7d8fca88', 'request_id': 'req-687dc847-f926-43d1-ae46-560d05b020a5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.834315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099627c2-6d87-483c-a8a4-335edd5f48ca admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:54.836080 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ced29d168e064480a2d3619a7d8fca88', 'request_id': 'req-687dc847-f926-43d1-ae46-560d05b020a5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.893903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ced29d168e064480a2d3619a7d8fca88', 'request_id': 'req-687dc847-f926-43d1-ae46-560d05b020a5'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.896222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb9014-27d9-4128-8524-5e9d607efb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:54.897511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb9014-27d9-4128-8524-5e9d607efb9e admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.897511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb9014-27d9-4128-8524-5e9d607efb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.897511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb9014-27d9-4128-8524-5e9d607efb9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.898093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb9014-27d9-4128-8524-5e9d607efb9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:54.900038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb9014-27d9-4128-8524-5e9d607efb9e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:54.909192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] The token cache is being invalidate because project ced29d168e064480a2d3619a7d8fca88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:54.909494 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.910206 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-687dc847-f926-43d1-ae46-560d05b020a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:54.913769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1088/2187] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:54 2022] DELETE /identity/v3/projects/ced29d168e064480a2d3619a7d8fca88 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:54.917886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928560a7-cd85-4a61-b7cd-6d587393d96c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.918162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928560a7-cd85-4a61-b7cd-6d587393d96c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.918660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928560a7-cd85-4a61-b7cd-6d587393d96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.939398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-928560a7-cd85-4a61-b7cd-6d587393d96c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:54.940633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-928560a7-cd85-4a61-b7cd-6d587393d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:54.951054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1089/2188] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:54.959617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaef4d5-b8d4-44b2-902f-3a76e4e16825 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:54.960315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaef4d5-b8d4-44b2-902f-3a76e4e16825 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:54.960745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaef4d5-b8d4-44b2-902f-3a76e4e16825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:54.982953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-deaef4d5-b8d4-44b2-902f-3a76e4e16825 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:54.984847 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-deaef4d5-b8d4-44b2-902f-3a76e4e16825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:55.003044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1090/2189] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:55.004674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaef4d5-b8d4-44b2-902f-3a76e4e16825 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.015544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaef4d5-b8d4-44b2-902f-3a76e4e16825 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.023149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1100/2190] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:54 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:55.024713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb9014-27d9-4128-8524-5e9d607efb9e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.025270 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd70de-8d65-4e2d-952a-765ac49f6ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.026300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd70de-8d65-4e2d-952a-765ac49f6ada admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.026516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd70de-8d65-4e2d-952a-765ac49f6ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.026903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd70de-8d65-4e2d-952a-765ac49f6ada admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.027017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2ddb9014-27d9-4128-8524-5e9d607efb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:55.027572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd70de-8d65-4e2d-952a-765ac49f6ada admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:55.027933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd70de-8d65-4e2d-952a-765ac49f6ada admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.030331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd70de-8d65-4e2d-952a-765ac49f6ada admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.048172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1091/2191] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:23:55.059492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd70de-8d65-4e2d-952a-765ac49f6ada admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.069642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd70de-8d65-4e2d-952a-765ac49f6ada admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.115997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb9014-27d9-4128-8524-5e9d607efb9e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.120857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688b1e0-ef8b-4195-bbf3-185d87fb24de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.121530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688b1e0-ef8b-4195-bbf3-185d87fb24de admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.121816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688b1e0-ef8b-4195-bbf3-185d87fb24de admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.122101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688b1e0-ef8b-4195-bbf3-185d87fb24de admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.122793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688b1e0-ef8b-4195-bbf3-185d87fb24de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.128273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688b1e0-ef8b-4195-bbf3-185d87fb24de admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.132238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60981a45-e0e0-4c20-a04b-f185e52c9ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.132861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60981a45-e0e0-4c20-a04b-f185e52c9ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.133139 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60981a45-e0e0-4c20-a04b-f185e52c9ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.133408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60981a45-e0e0-4c20-a04b-f185e52c9ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.134068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60981a45-e0e0-4c20-a04b-f185e52c9ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.135966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60981a45-e0e0-4c20-a04b-f185e52c9ea2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.203499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1101/2192] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:55 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:55.207673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688b1e0-ef8b-4195-bbf3-185d87fb24de admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.218277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688b1e0-ef8b-4195-bbf3-185d87fb24de admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.224898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2813f24-824d-4cd2-be29-f5746bc23a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.225782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2813f24-824d-4cd2-be29-f5746bc23a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.226170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2813f24-824d-4cd2-be29-f5746bc23a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.226547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2813f24-824d-4cd2-be29-f5746bc23a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.227355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2813f24-824d-4cd2-be29-f5746bc23a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.229190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2813f24-824d-4cd2-be29-f5746bc23a2a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.246328 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1102/2193] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:55.247022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2813f24-824d-4cd2-be29-f5746bc23a2a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.256455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1092/2194] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:55 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:55.263555 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60981a45-e0e0-4c20-a04b-f185e52c9ea2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.265043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2813f24-824d-4cd2-be29-f5746bc23a2a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.305891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:55.335032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4e76f-8eb3-464c-a309-a97e45ecb4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.335592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e76f-8eb3-464c-a309-a97e45ecb4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.335909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e76f-8eb3-464c-a309-a97e45ecb4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.336189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e76f-8eb3-464c-a309-a97e45ecb4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.336895 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4e76f-8eb3-464c-a309-a97e45ecb4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.338499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4e76f-8eb3-464c-a309-a97e45ecb4d8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.349227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1103/2195] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:55.353279 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4e76f-8eb3-464c-a309-a97e45ecb4d8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.363349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:55.363598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4e76f-8eb3-464c-a309-a97e45ecb4d8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.368667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17156eef-8d80-416b-8c44-e8abeef22ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.369057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60981a45-e0e0-4c20-a04b-f185e52c9ea2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.369768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17156eef-8d80-416b-8c44-e8abeef22ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.369987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17156eef-8d80-416b-8c44-e8abeef22ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.374251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17156eef-8d80-416b-8c44-e8abeef22ca9 admin admin] PATH_INFO: `/v3/projects/c70ade3d7f604830a6b48ba0386dce67/users/5aed2648ebea49559750faeb4c0006eb/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.376110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef8d56-c522-4c26-b35d-7a998c687553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.377079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef8d56-c522-4c26-b35d-7a998c687553 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.377521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef8d56-c522-4c26-b35d-7a998c687553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.378002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef8d56-c522-4c26-b35d-7a998c687553 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.379099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef8d56-c522-4c26-b35d-7a998c687553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.381812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef8d56-c522-4c26-b35d-7a998c687553 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.387544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17156eef-8d80-416b-8c44-e8abeef22ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c70ade3d7f604830a6b48ba0386dce67, user_id=5aed2648ebea49559750faeb4c0006eb, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.391379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17156eef-8d80-416b-8c44-e8abeef22ca9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.492775 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1104/2196] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:55 2022] PUT /identity/v3/projects/c70ade3d7f604830a6b48ba0386dce67/users/5aed2648ebea49559750faeb4c0006eb/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:23:55.493375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1093/2197] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:55 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:55.496572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17156eef-8d80-416b-8c44-e8abeef22ca9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.501209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f26dbe-892c-4f25-a84d-f488dca10b4f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.501497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f26dbe-892c-4f25-a84d-f488dca10b4f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.501797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f26dbe-892c-4f25-a84d-f488dca10b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.532929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-60f26dbe-892c-4f25-a84d-f488dca10b4f None None] MFA Rules not processed for user `5aed2648ebea49559750faeb4c0006eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:55.535298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-60f26dbe-892c-4f25-a84d-f488dca10b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:55.567970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17156eef-8d80-416b-8c44-e8abeef22ca9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.576877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a9a4b-2296-4426-89b9-947379c76a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.578288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9a4b-2296-4426-89b9-947379c76a2e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.578823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9a4b-2296-4426-89b9-947379c76a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.579172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a9a4b-2296-4426-89b9-947379c76a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.579988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a9a4b-2296-4426-89b9-947379c76a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.582229 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a9a4b-2296-4426-89b9-947379c76a2e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.602279 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1105/2198] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:55.603445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a9a4b-2296-4426-89b9-947379c76a2e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.617379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a9a4b-2296-4426-89b9-947379c76a2e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.622926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504a596-e484-4867-bdc2-88f33cb042eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.623947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504a596-e484-4867-bdc2-88f33cb042eb admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.624468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504a596-e484-4867-bdc2-88f33cb042eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.625342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504a596-e484-4867-bdc2-88f33cb042eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.626126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504a596-e484-4867-bdc2-88f33cb042eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.627954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504a596-e484-4867-bdc2-88f33cb042eb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.635835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1094/2199] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:55.645384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1106/2200] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:23:55.646766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f26dbe-892c-4f25-a84d-f488dca10b4f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.651805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504a596-e484-4867-bdc2-88f33cb042eb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.653572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2504a596-e484-4867-bdc2-88f33cb042eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:55.658460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f26dbe-892c-4f25-a84d-f488dca10b4f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.663428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c24962-f511-4a4b-bb71-aaae78f64a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.664150 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c24962-f511-4a4b-bb71-aaae78f64a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.664512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c24962-f511-4a4b-bb71-aaae78f64a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.664792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c24962-f511-4a4b-bb71-aaae78f64a53 admin admin] PATH_INFO: `/v3/projects/7e49a8a993484e33b0892a6a86d3d196/users/b07c458f1a5c4391aaaa79af49a236c7/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.680314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c24962-f511-4a4b-bb71-aaae78f64a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e49a8a993484e33b0892a6a86d3d196, user_id=b07c458f1a5c4391aaaa79af49a236c7, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.682978 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c24962-f511-4a4b-bb71-aaae78f64a53 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.714288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504a596-e484-4867-bdc2-88f33cb042eb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.718069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdf85b-b5ea-4c69-904a-f6b7c39385d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.718670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdf85b-b5ea-4c69-904a-f6b7c39385d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.718956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdf85b-b5ea-4c69-904a-f6b7c39385d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.719233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdf85b-b5ea-4c69-904a-f6b7c39385d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.729419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdf85b-b5ea-4c69-904a-f6b7c39385d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.730284 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdf85b-b5ea-4c69-904a-f6b7c39385d7 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.739771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1107/2201] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:23:55.783825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1095/2202] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:55 2022] PUT /identity/v3/projects/7e49a8a993484e33b0892a6a86d3d196/users/b07c458f1a5c4391aaaa79af49a236c7/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:55.788832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b041898-1361-4f59-a7d6-2ca1cc0ce84f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.789123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b041898-1361-4f59-a7d6-2ca1cc0ce84f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.789403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b041898-1361-4f59-a7d6-2ca1cc0ce84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.824069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7b041898-1361-4f59-a7d6-2ca1cc0ce84f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:55.825317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7b041898-1361-4f59-a7d6-2ca1cc0ce84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:55.856808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:55.898612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1108/2203] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:55.908242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c24962-f511-4a4b-bb71-aaae78f64a53 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:55.909948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-73c24962-f511-4a4b-bb71-aaae78f64a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:55.969764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c24962-f511-4a4b-bb71-aaae78f64a53 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:55.973487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08122b-9377-4e46-b86f-9d3100a0ac2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:55.974113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08122b-9377-4e46-b86f-9d3100a0ac2c service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:55.974398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08122b-9377-4e46-b86f-9d3100a0ac2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:55.974670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08122b-9377-4e46-b86f-9d3100a0ac2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:55.985875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08122b-9377-4e46-b86f-9d3100a0ac2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:55.987817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08122b-9377-4e46-b86f-9d3100a0ac2c service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:55.997769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1096/2204] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:56.946306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b041898-1361-4f59-a7d6-2ca1cc0ce84f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:56.948405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7b041898-1361-4f59-a7d6-2ca1cc0ce84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:56.966939 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b041898-1361-4f59-a7d6-2ca1cc0ce84f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:56.968278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:56.969567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:56.969846 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:56.970122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] PATH_INFO: `/v3/users/640ba0bfca094f3bbc130c2cddd6f2c0` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:56.971340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] RBAC: Authorizing `identity:delete_user(user_id=640ba0bfca094f3bbc130c2cddd6f2c0)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:56.973312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:57.198663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640ba0bfca094f3bbc130c2cddd6f2c0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:57.206292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640ba0bfca094f3bbc130c2cddd6f2c0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:57.285207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640ba0bfca094f3bbc130c2cddd6f2c0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:57.285837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640ba0bfca094f3bbc130c2cddd6f2c0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:57.300916 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640ba0bfca094f3bbc130c2cddd6f2c0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:57.310056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1109/2205] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:56 2022] DELETE /identity/v3/users/640ba0bfca094f3bbc130c2cddd6f2c0 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:57.314488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8ca68-4edc-4434-8510-91e47de46119 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:57.315013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8ca68-4edc-4434-8510-91e47de46119 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:57.315122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8ca68-4edc-4434-8510-91e47de46119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:57.333992 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3cc8ca68-4edc-4434-8510-91e47de46119 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:57.335205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3cc8ca68-4edc-4434-8510-91e47de46119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:57.393707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:57.405419 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1097/2206] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:57.415747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:57.420934 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:57.480458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804066c-46dc-46d5-b1da-415862b3483c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:57.486478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9065dd-4827-4d0b-8b27-c6c515386d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:57.486478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9065dd-4827-4d0b-8b27-c6c515386d80 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:57.486478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9065dd-4827-4d0b-8b27-c6c515386d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:57.486478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9065dd-4827-4d0b-8b27-c6c515386d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:57.496290 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9065dd-4827-4d0b-8b27-c6c515386d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:57.497757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9065dd-4827-4d0b-8b27-c6c515386d80 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:57.509765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1110/2207] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:57.683947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358107d-87d9-4694-ac75-435bd3d22961 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:57.684243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358107d-87d9-4694-ac75-435bd3d22961 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:57.684512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358107d-87d9-4694-ac75-435bd3d22961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:57.716954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3358107d-87d9-4694-ac75-435bd3d22961 None None] MFA Rules not processed for user `5aed2648ebea49559750faeb4c0006eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:57.720546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3358107d-87d9-4694-ac75-435bd3d22961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:57.884162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1098/2208] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:58.128142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9065dd-4827-4d0b-8b27-c6c515386d80 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:58.130303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ef9065dd-4827-4d0b-8b27-c6c515386d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:58.185208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9065dd-4827-4d0b-8b27-c6c515386d80 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:58.192982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:58.193701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:58.194041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:58.195947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] PATH_INFO: `/v3/projects/2935fa5f518c4d5c8a05a5677770d1cc` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:58.197658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2935fa5f518c4d5c8a05a5677770d1cc)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:58.200946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:58.345714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:58.352935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2935fa5f518c4d5c8a05a5677770d1cc', 'request_id': 'req-d556c328-c3ba-4da7-a7c0-216316e4952c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:58.383058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2935fa5f518c4d5c8a05a5677770d1cc', 'request_id': 'req-d556c328-c3ba-4da7-a7c0-216316e4952c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:58.384011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2935fa5f518c4d5c8a05a5677770d1cc', 'request_id': 'req-d556c328-c3ba-4da7-a7c0-216316e4952c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:58.485321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] The token cache is being invalidate because project 2935fa5f518c4d5c8a05a5677770d1cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:23:58.485630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:58.486354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d556c328-c3ba-4da7-a7c0-216316e4952c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:23:58.488219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1111/2209] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:23:58 2022] DELETE /identity/v3/projects/2935fa5f518c4d5c8a05a5677770d1cc => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:58.493459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6029d5b-e8d2-4693-8a97-21586983848c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:58.494261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6029d5b-e8d2-4693-8a97-21586983848c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:58.494546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6029d5b-e8d2-4693-8a97-21586983848c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:58.531814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f6029d5b-e8d2-4693-8a97-21586983848c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:58.531814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f6029d5b-e8d2-4693-8a97-21586983848c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:58.647114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1099/2210] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:23:58.653312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4067b-9faf-4ed5-95cf-2307a53175ff None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:58.653819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4067b-9faf-4ed5-95cf-2307a53175ff None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:58.654202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4067b-9faf-4ed5-95cf-2307a53175ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:58.689789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bdf4067b-9faf-4ed5-95cf-2307a53175ff None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:58.699891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bdf4067b-9faf-4ed5-95cf-2307a53175ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:58.713157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1112/2211] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:58.719227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6029d5b-e8d2-4693-8a97-21586983848c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:58.753693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6029d5b-e8d2-4693-8a97-21586983848c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:58.759532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f4d70-1f4a-4618-b690-65e07a34ff6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:58.760235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f4d70-1f4a-4618-b690-65e07a34ff6f admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:58.760937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f4d70-1f4a-4618-b690-65e07a34ff6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:58.760937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f4d70-1f4a-4618-b690-65e07a34ff6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:58.767722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f4d70-1f4a-4618-b690-65e07a34ff6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:23:58.767722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f4d70-1f4a-4618-b690-65e07a34ff6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:58.767722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f4d70-1f4a-4618-b690-65e07a34ff6f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:58.782348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1100/2212] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:23:58.784884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4067b-9faf-4ed5-95cf-2307a53175ff None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:58.806723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4067b-9faf-4ed5-95cf-2307a53175ff None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:58.813818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d46919-4b3b-411a-9990-637dbcd3df56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:58.814531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d46919-4b3b-411a-9990-637dbcd3df56 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:58.814878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d46919-4b3b-411a-9990-637dbcd3df56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:58.815239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d46919-4b3b-411a-9990-637dbcd3df56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:58.816037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d46919-4b3b-411a-9990-637dbcd3df56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:58.818178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d46919-4b3b-411a-9990-637dbcd3df56 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:58.917038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1113/2213] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:23:58 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:23:58.922712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f4d70-1f4a-4618-b690-65e07a34ff6f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:58.953178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f4d70-1f4a-4618-b690-65e07a34ff6f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:59.069606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d00e1d-bb80-4360-a8e9-dc6a86317ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:59.070360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d00e1d-bb80-4360-a8e9-dc6a86317ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:59.075226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d00e1d-bb80-4360-a8e9-dc6a86317ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:59.075226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d00e1d-bb80-4360-a8e9-dc6a86317ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:59.075226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d00e1d-bb80-4360-a8e9-dc6a86317ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:59.075226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d00e1d-bb80-4360-a8e9-dc6a86317ea0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:59.199959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1101/2214] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:23:58 2022] POST /identity/v3/users => generated 286 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:23:59.202390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d46919-4b3b-411a-9990-637dbcd3df56 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:59.218397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d46919-4b3b-411a-9990-637dbcd3df56 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:59.225474 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41d07f-33dc-46ce-9695-6622e9780b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:59.226591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41d07f-33dc-46ce-9695-6622e9780b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:59.228213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41d07f-33dc-46ce-9695-6622e9780b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:59.228657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41d07f-33dc-46ce-9695-6622e9780b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:59.230139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41d07f-33dc-46ce-9695-6622e9780b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:59.233080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41d07f-33dc-46ce-9695-6622e9780b05 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:59.258196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d00e1d-bb80-4360-a8e9-dc6a86317ea0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:59.277420 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d00e1d-bb80-4360-a8e9-dc6a86317ea0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:59.284724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48900f48-95cf-46a9-bf85-692b85b27b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:59.285435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48900f48-95cf-46a9-bf85-692b85b27b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:59.285771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48900f48-95cf-46a9-bf85-692b85b27b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:59.286433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1114/2215] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:59.287204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48900f48-95cf-46a9-bf85-692b85b27b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:59.288248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48900f48-95cf-46a9-bf85-692b85b27b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:59.290232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48900f48-95cf-46a9-bf85-692b85b27b04 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:59.314447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1102/2216] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:23:59.319171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41d07f-33dc-46ce-9695-6622e9780b05 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:59.338648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41d07f-33dc-46ce-9695-6622e9780b05 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:59.356232 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41028caa-d92f-4a89-b05e-d0b21e0dae90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:59.359048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41028caa-d92f-4a89-b05e-d0b21e0dae90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:59.359048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41028caa-d92f-4a89-b05e-d0b21e0dae90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:59.361529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41028caa-d92f-4a89-b05e-d0b21e0dae90 admin admin] PATH_INFO: `/v3/projects/fc4141860179465eb3db2ad4b3c16eec/users/736713aabecd425087e1fa8bfebb368a/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:59.385362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41028caa-d92f-4a89-b05e-d0b21e0dae90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4141860179465eb3db2ad4b3c16eec, user_id=736713aabecd425087e1fa8bfebb368a, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:59.388613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41028caa-d92f-4a89-b05e-d0b21e0dae90 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:59.472869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1115/2217] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:23:59 2022] PUT /identity/v3/projects/fc4141860179465eb3db2ad4b3c16eec/users/736713aabecd425087e1fa8bfebb368a/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:23:59.485487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026af122-3f7a-45dc-a934-88bd56ee82d9 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:59.485889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026af122-3f7a-45dc-a934-88bd56ee82d9 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:59.486266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026af122-3f7a-45dc-a934-88bd56ee82d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:59.507044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:23:59.563030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-026af122-3f7a-45dc-a934-88bd56ee82d9 None None] MFA Rules not processed for user `736713aabecd425087e1fa8bfebb368a`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:23:59.565440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-026af122-3f7a-45dc-a934-88bd56ee82d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:59.665501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1103/2218] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:23:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:23:59.682011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41028caa-d92f-4a89-b05e-d0b21e0dae90 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:59.683936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-41028caa-d92f-4a89-b05e-d0b21e0dae90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:59.766966 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41028caa-d92f-4a89-b05e-d0b21e0dae90 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:59.771263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8b5d1-52f5-4458-96c4-aa4fc6e6fb56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a234a96eccd4f38b0213d7d0461ad24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:59.772744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8b5d1-52f5-4458-96c4-aa4fc6e6fb56 service glance] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:59.773098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8b5d1-52f5-4458-96c4-aa4fc6e6fb56 service glance] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:59.773454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8b5d1-52f5-4458-96c4-aa4fc6e6fb56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:59.785880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8b5d1-52f5-4458-96c4-aa4fc6e6fb56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:59.787537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8b5d1-52f5-4458-96c4-aa4fc6e6fb56 service glance] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:23:59.801650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1116/2219] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:23:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:23:59.843019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026af122-3f7a-45dc-a934-88bd56ee82d9 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:23:59.845794 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-026af122-3f7a-45dc-a934-88bd56ee82d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:23:59.935907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026af122-3f7a-45dc-a934-88bd56ee82d9 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:23:59.949305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:23:59.950170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:23:59.950453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:23:59.950730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] PATH_INFO: `/v3/users/4482913e75f1480ab35e723124aa44ac` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:23:59.952588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4482913e75f1480ab35e723124aa44ac)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:23:59.955384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:00.256894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4482913e75f1480ab35e723124aa44ac'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:00.275322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4482913e75f1480ab35e723124aa44ac'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:00.350169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4482913e75f1480ab35e723124aa44ac'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:00.352416 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4482913e75f1480ab35e723124aa44ac'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:00.354133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4482913e75f1480ab35e723124aa44ac'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:00.403744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1104/2220] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:23:59 2022] DELETE /identity/v3/users/4482913e75f1480ab35e723124aa44ac => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:00.408813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59316e-b22c-4263-872b-d4c6a5510229 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:00.409283 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59316e-b22c-4263-872b-d4c6a5510229 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:00.415549 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59316e-b22c-4263-872b-d4c6a5510229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:00.466011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9c59316e-b22c-4263-872b-d4c6a5510229 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:00.472325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9c59316e-b22c-4263-872b-d4c6a5510229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:00.677301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1117/2221] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:00.687685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:00.690003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:00.705447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:00.756069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b0535-275c-4d89-8ad3-87e7ea8add8b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:00.759951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedbeb0-636a-421e-a62d-c19886d82319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:00.761119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedbeb0-636a-421e-a62d-c19886d82319 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:00.767112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedbeb0-636a-421e-a62d-c19886d82319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:00.767112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedbeb0-636a-421e-a62d-c19886d82319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:00.774257 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedbeb0-636a-421e-a62d-c19886d82319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:00.776706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedbeb0-636a-421e-a62d-c19886d82319 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:00.806662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1105/2222] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:01.266273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59316e-b22c-4263-872b-d4c6a5510229 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:01.267685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9c59316e-b22c-4263-872b-d4c6a5510229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:01.280552 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59316e-b22c-4263-872b-d4c6a5510229 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:01.286320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:01.286979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:01.287260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:01.287538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] PATH_INFO: `/v3/projects/0cbb6f2bc24f46749e888bbf98906330` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:01.291131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cbb6f2bc24f46749e888bbf98906330)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:01.291131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:01.297704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:01.372730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedbeb0-636a-421e-a62d-c19886d82319 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:01.374244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bdedbeb0-636a-421e-a62d-c19886d82319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:01.388972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedbeb0-636a-421e-a62d-c19886d82319 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:01.414745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cbb6f2bc24f46749e888bbf98906330', 'request_id': 'req-c454c626-f8fb-4bd8-917a-c551c6b85a4b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.423209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cbb6f2bc24f46749e888bbf98906330', 'request_id': 'req-c454c626-f8fb-4bd8-917a-c551c6b85a4b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.429993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cbb6f2bc24f46749e888bbf98906330', 'request_id': 'req-c454c626-f8fb-4bd8-917a-c551c6b85a4b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.459406 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:01.460194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:01.460546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:01.460891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] PATH_INFO: `/v3/users/fa8f8aea6be94cd2bf47f18cc0116364` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:01.462134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa8f8aea6be94cd2bf47f18cc0116364)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:01.464230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:01.505130 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] The token cache is being invalidate because project 0cbb6f2bc24f46749e888bbf98906330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:01.505529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.506465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.508174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1118/2223] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:01 2022] DELETE /identity/v3/projects/0cbb6f2bc24f46749e888bbf98906330 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:01.511977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:01.513629 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:01.536167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454c626-f8fb-4bd8-917a-c551c6b85a4b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:01.542257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:01.543113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:01.543742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:01.544024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] PATH_INFO: `/v3/users/d04069adb90e42798f6d3a79c1dc28b2` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:01.545435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d04069adb90e42798f6d3a79c1dc28b2)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:01.548123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:01.692358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa8f8aea6be94cd2bf47f18cc0116364'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.714017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa8f8aea6be94cd2bf47f18cc0116364'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.811357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa8f8aea6be94cd2bf47f18cc0116364'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.812123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa8f8aea6be94cd2bf47f18cc0116364'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.812890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa8f8aea6be94cd2bf47f18cc0116364'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.836275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd04069adb90e42798f6d3a79c1dc28b2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.836481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1106/2224] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:01 2022] DELETE /identity/v3/users/fa8f8aea6be94cd2bf47f18cc0116364 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:24:01.864722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd04069adb90e42798f6d3a79c1dc28b2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.983643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd04069adb90e42798f6d3a79c1dc28b2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:01.984529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd04069adb90e42798f6d3a79c1dc28b2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:02.004017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd04069adb90e42798f6d3a79c1dc28b2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:02.017105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1119/2225] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:01 2022] DELETE /identity/v3/users/d04069adb90e42798f6d3a79c1dc28b2 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:24:02.204585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:02.206509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:02.262502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:02.295711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a3af4-bf35-4289-8c0e-3cd17f2e64bd admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:02.370900 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:02.371878 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:02.372869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:02.373302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] PATH_INFO: `/v3/projects/2129d74b666f4144af582be0e24e2027` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:02.375565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=2129d74b666f4144af582be0e24e2027)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:02.379366 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:02.527341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2129d74b666f4144af582be0e24e2027', 'request_id': 'req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:02.548425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2129d74b666f4144af582be0e24e2027', 'request_id': 'req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:02.625662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2129d74b666f4144af582be0e24e2027', 'request_id': 'req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:02.638906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] The token cache is being invalidate because project 2129d74b666f4144af582be0e24e2027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:02.639397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:02.640818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:02.647244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1107/2226] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:02 2022] DELETE /identity/v3/projects/2129d74b666f4144af582be0e24e2027 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:02.648656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:02.652930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:02.674151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587c51a-546e-4151-92f2-294028e16c7a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:02.774931 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:02.775668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:02.775982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:02.776265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] PATH_INFO: `/v3/users/eb2a8b66a4ef452ab3a84910841155aa` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:02.778734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2a8b66a4ef452ab3a84910841155aa)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:02.781293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:02.791721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:03.067189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2a8b66a4ef452ab3a84910841155aa'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:03.067359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:03.069318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:03.085128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2a8b66a4ef452ab3a84910841155aa'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:03.097821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ed764-eceb-4bc6-a0bc-742f5d5be8ee admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:03.115096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:03.117004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:03.117446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:03.117872 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] PATH_INFO: `/v3/projects/7f30ceea6ec742c08afcf636c51a7e65` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:03.119437 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f30ceea6ec742c08afcf636c51a7e65)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:03.126446 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:03.161185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2a8b66a4ef452ab3a84910841155aa'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:03.161844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2a8b66a4ef452ab3a84910841155aa'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:03.181150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2a8b66a4ef452ab3a84910841155aa'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:03.203752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1120/2227] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:02 2022] DELETE /identity/v3/users/eb2a8b66a4ef452ab3a84910841155aa => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:03.267226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f30ceea6ec742c08afcf636c51a7e65', 'request_id': 'req-14043353-77e4-46d4-afe8-ef2398b60086'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:03.276637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f30ceea6ec742c08afcf636c51a7e65', 'request_id': 'req-14043353-77e4-46d4-afe8-ef2398b60086'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:03.373927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f30ceea6ec742c08afcf636c51a7e65', 'request_id': 'req-14043353-77e4-46d4-afe8-ef2398b60086'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:03.392984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] The token cache is being invalidate because project 7f30ceea6ec742c08afcf636c51a7e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:03.393521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:03.394774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:03.400930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:03.402513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:03.404258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1108/2228] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:03 2022] DELETE /identity/v3/projects/7f30ceea6ec742c08afcf636c51a7e65 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:24:03.499302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f633e40-9d0a-4ce7-9e20-6f45d44ab911 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:03.553167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:03.599072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:03.600107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:03.602347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:03.602347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] PATH_INFO: `/v3/users/a78b3ae46f8d4c549cba9d2a17204fa9` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:03.603889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a78b3ae46f8d4c549cba9d2a17204fa9)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:03.606147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:03.703485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:03.706034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:03.723683 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14043353-77e4-46d4-afe8-ef2398b60086 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:03.733092 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:03.734174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:03.734614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:03.734889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] PATH_INFO: `/v3/projects/ffacd75c314e42bdb90754176d3f0a7f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:03.749113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffacd75c314e42bdb90754176d3f0a7f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:03.749113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:03.914553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:04.024641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a78b3ae46f8d4c549cba9d2a17204fa9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:04.024641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a78b3ae46f8d4c549cba9d2a17204fa9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:04.188460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffacd75c314e42bdb90754176d3f0a7f', 'request_id': 'req-5eb6713a-5ce6-4945-afb4-e40854fa2605'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:04.194509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffacd75c314e42bdb90754176d3f0a7f', 'request_id': 'req-5eb6713a-5ce6-4945-afb4-e40854fa2605'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:04.281587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a78b3ae46f8d4c549cba9d2a17204fa9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:04.281587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a78b3ae46f8d4c549cba9d2a17204fa9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:04.316071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-070895d0-59d1-4354-924a-567abeda2fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a78b3ae46f8d4c549cba9d2a17204fa9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:04.323320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1121/2229] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:03 2022] DELETE /identity/v3/users/a78b3ae46f8d4c549cba9d2a17204fa9 => generated 0 bytes in 923 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:04.384367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffacd75c314e42bdb90754176d3f0a7f', 'request_id': 'req-5eb6713a-5ce6-4945-afb4-e40854fa2605'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:04.419625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] The token cache is being invalidate because project ffacd75c314e42bdb90754176d3f0a7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:04.420760 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:04.421750 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5eb6713a-5ce6-4945-afb4-e40854fa2605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:04.432484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1109/2230] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:03 2022] DELETE /identity/v3/projects/ffacd75c314e42bdb90754176d3f0a7f => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:04.433925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241af7f-e07f-4aff-a20c-dbc4dfe2bf16 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:04.434456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241af7f-e07f-4aff-a20c-dbc4dfe2bf16 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:04.434936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241af7f-e07f-4aff-a20c-dbc4dfe2bf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:04.467442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2241af7f-e07f-4aff-a20c-dbc4dfe2bf16 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:04.469710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2241af7f-e07f-4aff-a20c-dbc4dfe2bf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:04.592995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1122/2231] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:04.603045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c18278-e3ab-49fe-b7e5-7f18d2f32d43 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:04.608644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c18278-e3ab-49fe-b7e5-7f18d2f32d43 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:04.608644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c18278-e3ab-49fe-b7e5-7f18d2f32d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:04.632003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-55c18278-e3ab-49fe-b7e5-7f18d2f32d43 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:04.634723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-55c18278-e3ab-49fe-b7e5-7f18d2f32d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:04.674938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1110/2232] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:04.680089 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241af7f-e07f-4aff-a20c-dbc4dfe2bf16 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:04.696737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241af7f-e07f-4aff-a20c-dbc4dfe2bf16 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:04.724516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0251975-4b6a-4514-90b6-99cf491616e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:04.725559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0251975-4b6a-4514-90b6-99cf491616e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:04.726015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0251975-4b6a-4514-90b6-99cf491616e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:04.726449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0251975-4b6a-4514-90b6-99cf491616e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:04.727319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0251975-4b6a-4514-90b6-99cf491616e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:04.730596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0251975-4b6a-4514-90b6-99cf491616e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:04.739272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0251975-4b6a-4514-90b6-99cf491616e5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:04.775002 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1123/2233] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:24:04.779581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c18278-e3ab-49fe-b7e5-7f18d2f32d43 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:04.806255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c18278-e3ab-49fe-b7e5-7f18d2f32d43 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:04.807682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cb72b-046d-4890-98f6-b1d5b97d7903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:04.808358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cb72b-046d-4890-98f6-b1d5b97d7903 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:04.808643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cb72b-046d-4890-98f6-b1d5b97d7903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:04.808919 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cb72b-046d-4890-98f6-b1d5b97d7903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:04.809601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cb72b-046d-4890-98f6-b1d5b97d7903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:04.816393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cb72b-046d-4890-98f6-b1d5b97d7903 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:05.000458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0251975-4b6a-4514-90b6-99cf491616e5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:05.005421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1111/2234] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:04 2022] POST /identity/v3/projects => generated 381 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:05.011815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cb72b-046d-4890-98f6-b1d5b97d7903 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:05.012298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f0251975-4b6a-4514-90b6-99cf491616e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:05.034794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0251975-4b6a-4514-90b6-99cf491616e5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:05.121753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:05.122728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.123702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.124476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] PATH_INFO: `/v3/projects/34e9f79920644d838392fcffe4854035` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.130642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=34e9f79920644d838392fcffe4854035)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:05.133593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:05.171337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cb72b-046d-4890-98f6-b1d5b97d7903 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:05.184823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28fff9-aa7a-4f51-abd1-d2691b279ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:05.184823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28fff9-aa7a-4f51-abd1-d2691b279ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.184823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28fff9-aa7a-4f51-abd1-d2691b279ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.184823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28fff9-aa7a-4f51-abd1-d2691b279ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.185318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28fff9-aa7a-4f51-abd1-d2691b279ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:05.186047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28fff9-aa7a-4f51-abd1-d2691b279ed7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:05.311434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34e9f79920644d838392fcffe4854035', 'request_id': 'req-88e51d2c-f771-4d45-b5b5-b085183c2b6f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:05.322043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34e9f79920644d838392fcffe4854035', 'request_id': 'req-88e51d2c-f771-4d45-b5b5-b085183c2b6f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:05.322555 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1112/2235] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:05 2022] POST /identity/v3/users => generated 291 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:05.328975 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34e9f79920644d838392fcffe4854035', 'request_id': 'req-88e51d2c-f771-4d45-b5b5-b085183c2b6f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:05.332984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28fff9-aa7a-4f51-abd1-d2691b279ed7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:05.358712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28fff9-aa7a-4f51-abd1-d2691b279ed7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:05.412969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:05.419864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] The token cache is being invalidate because project 34e9f79920644d838392fcffe4854035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:05.419864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:05.422839 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88e51d2c-f771-4d45-b5b5-b085183c2b6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:05.425172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1124/2236] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:04 2022] DELETE /identity/v3/projects/34e9f79920644d838392fcffe4854035 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:24:05.428956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4064b7a-aaed-4d50-bd54-f4d39b630aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:05.428956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4064b7a-aaed-4d50-bd54-f4d39b630aeb admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.428956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4064b7a-aaed-4d50-bd54-f4d39b630aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.428956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4064b7a-aaed-4d50-bd54-f4d39b630aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.429340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4064b7a-aaed-4d50-bd54-f4d39b630aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:05.430443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4064b7a-aaed-4d50-bd54-f4d39b630aeb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:05.433167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc531964-f594-45a6-a203-7387fed0c4ea None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.433664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc531964-f594-45a6-a203-7387fed0c4ea None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.434113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc531964-f594-45a6-a203-7387fed0c4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.443977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1113/2237] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:05.447075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4064b7a-aaed-4d50-bd54-f4d39b630aeb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:05.450230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f4064b7a-aaed-4d50-bd54-f4d39b630aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:05.464860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4064b7a-aaed-4d50-bd54-f4d39b630aeb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:05.465644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bc531964-f594-45a6-a203-7387fed0c4ea None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:05.470727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cf55c-b8a5-46b0-8001-6563824039c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:05.471385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cf55c-b8a5-46b0-8001-6563824039c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.471717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cf55c-b8a5-46b0-8001-6563824039c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.472052 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cf55c-b8a5-46b0-8001-6563824039c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.472959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cf55c-b8a5-46b0-8001-6563824039c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:05.473300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bc531964-f594-45a6-a203-7387fed0c4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:05.477386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cf55c-b8a5-46b0-8001-6563824039c3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:05.487233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1125/2238] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:05.491913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1114/2239] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:05.495660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813d2f-104b-41e3-b814-c9f2aaf3bb6f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.495889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cf55c-b8a5-46b0-8001-6563824039c3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:05.495979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813d2f-104b-41e3-b814-c9f2aaf3bb6f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.496223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813d2f-104b-41e3-b814-c9f2aaf3bb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.509526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cf55c-b8a5-46b0-8001-6563824039c3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:05.521818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a1bc1-4d94-4771-884e-dd2c8acc6b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:05.523321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a1bc1-4d94-4771-884e-dd2c8acc6b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.524327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a1bc1-4d94-4771-884e-dd2c8acc6b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.524801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a1bc1-4d94-4771-884e-dd2c8acc6b1a admin admin] PATH_INFO: `/v3/projects/4bd65d186e3f41cc9541660aee8b1e52/users/ce5998047c5b449c9573e6f4b5ade27e/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.550288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3b813d2f-104b-41e3-b814-c9f2aaf3bb6f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:05.551562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3b813d2f-104b-41e3-b814-c9f2aaf3bb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:05.562886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a1bc1-4d94-4771-884e-dd2c8acc6b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd65d186e3f41cc9541660aee8b1e52, user_id=ce5998047c5b449c9573e6f4b5ade27e, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:05.568741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a1bc1-4d94-4771-884e-dd2c8acc6b1a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:05.586489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1126/2240] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:05.608639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813d2f-104b-41e3-b814-c9f2aaf3bb6f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:05.621308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813d2f-104b-41e3-b814-c9f2aaf3bb6f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:05.633503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeae991-1e1a-4cf1-99c9-f1a794509cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:05.636910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeae991-1e1a-4cf1-99c9-f1a794509cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.637372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeae991-1e1a-4cf1-99c9-f1a794509cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.637812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeae991-1e1a-4cf1-99c9-f1a794509cc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.638744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeae991-1e1a-4cf1-99c9-f1a794509cc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:05.640968 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeae991-1e1a-4cf1-99c9-f1a794509cc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:05.643011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeae991-1e1a-4cf1-99c9-f1a794509cc9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:05.667990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1127/2241] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:24:05.668189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1115/2242] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:05 2022] PUT /identity/v3/projects/4bd65d186e3f41cc9541660aee8b1e52/users/ce5998047c5b449c9573e6f4b5ade27e/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:05.678072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c58bf2-1787-40d1-987b-c6c5e59917c5 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.678618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeae991-1e1a-4cf1-99c9-f1a794509cc9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:05.679335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c58bf2-1787-40d1-987b-c6c5e59917c5 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.683880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c58bf2-1787-40d1-987b-c6c5e59917c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.699065 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeae991-1e1a-4cf1-99c9-f1a794509cc9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:05.732987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b1c58bf2-1787-40d1-987b-c6c5e59917c5 None None] MFA Rules not processed for user `ce5998047c5b449c9573e6f4b5ade27e`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:05.738462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b1c58bf2-1787-40d1-987b-c6c5e59917c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:05.758550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:05.807096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ab364-ffe9-48f2-8761-0e3c354533a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:05.807952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ab364-ffe9-48f2-8761-0e3c354533a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.808359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ab364-ffe9-48f2-8761-0e3c354533a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.808826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ab364-ffe9-48f2-8761-0e3c354533a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.809999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ab364-ffe9-48f2-8761-0e3c354533a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:05.812322 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ab364-ffe9-48f2-8761-0e3c354533a5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:05.831841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1116/2243] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:05.834609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c58bf2-1787-40d1-987b-c6c5e59917c5 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:05.849121 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c58bf2-1787-40d1-987b-c6c5e59917c5 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:05.856985 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938342d-78af-4db8-8116-37dfdd24c96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:05.857716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938342d-78af-4db8-8116-37dfdd24c96f admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:05.859922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938342d-78af-4db8-8116-37dfdd24c96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:05.860263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938342d-78af-4db8-8116-37dfdd24c96f admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:05.861294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938342d-78af-4db8-8116-37dfdd24c96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:05.870165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938342d-78af-4db8-8116-37dfdd24c96f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:05.939056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1128/2244] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:05 2022] POST /identity/v3/projects => generated 385 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:05.943585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ab364-ffe9-48f2-8761-0e3c354533a5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.013694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ab364-ffe9-48f2-8761-0e3c354533a5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:06.020227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b198f1-d90d-435b-a6b6-f40a97f546a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:06.020860 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b198f1-d90d-435b-a6b6-f40a97f546a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.021140 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b198f1-d90d-435b-a6b6-f40a97f546a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.021415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b198f1-d90d-435b-a6b6-f40a97f546a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.022084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b198f1-d90d-435b-a6b6-f40a97f546a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:06.024632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b198f1-d90d-435b-a6b6-f40a97f546a5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:06.059362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1117/2245] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:05 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:06.061234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac619057-7ed8-410b-b899-cdf794279bb7 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.061678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac619057-7ed8-410b-b899-cdf794279bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.062168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac619057-7ed8-410b-b899-cdf794279bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.098115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ac619057-7ed8-410b-b899-cdf794279bb7 None None] MFA Rules not processed for user `b07c458f1a5c4391aaaa79af49a236c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:06.099495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ac619057-7ed8-410b-b899-cdf794279bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:06.167266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1118/2246] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:06.168155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac619057-7ed8-410b-b899-cdf794279bb7 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.180405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1129/2247] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:05 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:24:06.184103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b198f1-d90d-435b-a6b6-f40a97f546a5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.197874 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b198f1-d90d-435b-a6b6-f40a97f546a5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:06.259662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac619057-7ed8-410b-b899-cdf794279bb7 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:06.266226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d45602-61e8-4bd6-9c37-e128872c4955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:06.267073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d45602-61e8-4bd6-9c37-e128872c4955 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.267513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d45602-61e8-4bd6-9c37-e128872c4955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.267963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d45602-61e8-4bd6-9c37-e128872c4955 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.268767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d45602-61e8-4bd6-9c37-e128872c4955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:06.271224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d45602-61e8-4bd6-9c37-e128872c4955 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:06.325632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcedca1-86c2-4044-bc23-b8231fac3013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:06.328999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcedca1-86c2-4044-bc23-b8231fac3013 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.329361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcedca1-86c2-4044-bc23-b8231fac3013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.329803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcedca1-86c2-4044-bc23-b8231fac3013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.330593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcedca1-86c2-4044-bc23-b8231fac3013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:06.333994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcedca1-86c2-4044-bc23-b8231fac3013 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:06.348392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1130/2248] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:06.352508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcedca1-86c2-4044-bc23-b8231fac3013 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.354469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2fcedca1-86c2-4044-bc23-b8231fac3013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:06.377486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1119/2249] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:06 2022] POST /identity/v3/users => generated 298 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:06.379524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d45602-61e8-4bd6-9c37-e128872c4955 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.400276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d45602-61e8-4bd6-9c37-e128872c4955 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:06.409042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a21b1-e55e-4ee1-982d-40ee0fc7d114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:06.410085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a21b1-e55e-4ee1-982d-40ee0fc7d114 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.412114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a21b1-e55e-4ee1-982d-40ee0fc7d114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.412502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a21b1-e55e-4ee1-982d-40ee0fc7d114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.414582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a21b1-e55e-4ee1-982d-40ee0fc7d114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:06.428421 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a21b1-e55e-4ee1-982d-40ee0fc7d114 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:06.456238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1120/2250] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:06.458865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a21b1-e55e-4ee1-982d-40ee0fc7d114 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.474343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a21b1-e55e-4ee1-982d-40ee0fc7d114 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:06.481381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888376d-c0d8-4f76-b88b-3429a191f653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:06.482374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888376d-c0d8-4f76-b88b-3429a191f653 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.482860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888376d-c0d8-4f76-b88b-3429a191f653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.483326 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888376d-c0d8-4f76-b88b-3429a191f653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.487502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888376d-c0d8-4f76-b88b-3429a191f653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:06.492012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888376d-c0d8-4f76-b88b-3429a191f653 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:06.505634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1121/2251] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:06.506789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888376d-c0d8-4f76-b88b-3429a191f653 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.520682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcedca1-86c2-4044-bc23-b8231fac3013 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:06.525040 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888376d-c0d8-4f76-b88b-3429a191f653 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:06.525861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c6ac8-3eeb-40db-91c3-e79ac1a6e122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:06.526574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c6ac8-3eeb-40db-91c3-e79ac1a6e122 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.527548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c6ac8-3eeb-40db-91c3-e79ac1a6e122 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.527962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c6ac8-3eeb-40db-91c3-e79ac1a6e122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.535041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9a3bb-4beb-44fd-ba10-8ce70898d32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:06.535931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9a3bb-4beb-44fd-ba10-8ce70898d32b admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.536314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9a3bb-4beb-44fd-ba10-8ce70898d32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.536752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9a3bb-4beb-44fd-ba10-8ce70898d32b admin admin] PATH_INFO: `/v3/projects/a3615c2b50b341508e01f54bf81e6eb6/users/1584f19f761549dc9b40ead80ea14224/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.541565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c6ac8-3eeb-40db-91c3-e79ac1a6e122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:06.544280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c6ac8-3eeb-40db-91c3-e79ac1a6e122 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:06.554460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9a3bb-4beb-44fd-ba10-8ce70898d32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3615c2b50b341508e01f54bf81e6eb6, user_id=1584f19f761549dc9b40ead80ea14224, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:06.557986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b9a3bb-4beb-44fd-ba10-8ce70898d32b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:06.568112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1131/2252] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:06.569114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c6ac8-3eeb-40db-91c3-e79ac1a6e122 service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.590230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c6ac8-3eeb-40db-91c3-e79ac1a6e122 service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:06.602182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e87883-b1f8-4c28-a887-54c94f24c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:06.606236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e87883-b1f8-4c28-a887-54c94f24c7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.606236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e87883-b1f8-4c28-a887-54c94f24c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.606236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e87883-b1f8-4c28-a887-54c94f24c7c1 admin admin] PATH_INFO: `/v3/projects/7f855614e08944aa8c780366cfe3fd17/users/9c5af3de8f1b4671a023bc84594b7dcc/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.626877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e87883-b1f8-4c28-a887-54c94f24c7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f855614e08944aa8c780366cfe3fd17, user_id=9c5af3de8f1b4671a023bc84594b7dcc, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:06.630055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e87883-b1f8-4c28-a887-54c94f24c7c1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:06.636815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1122/2253] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:06 2022] PUT /identity/v3/projects/a3615c2b50b341508e01f54bf81e6eb6/users/1584f19f761549dc9b40ead80ea14224/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:06.650456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505817e5-e790-460e-aa44-957eccfc3a10 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.652248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505817e5-e790-460e-aa44-957eccfc3a10 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.660200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505817e5-e790-460e-aa44-957eccfc3a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.705621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-505817e5-e790-460e-aa44-957eccfc3a10 None None] MFA Rules not processed for user `1584f19f761549dc9b40ead80ea14224`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:06.707740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-505817e5-e790-460e-aa44-957eccfc3a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:06.740663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1132/2254] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:06 2022] PUT /identity/v3/projects/7f855614e08944aa8c780366cfe3fd17/users/9c5af3de8f1b4671a023bc84594b7dcc/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:06.748906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e87883-b1f8-4c28-a887-54c94f24c7c1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.837964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1123/2255] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:06.861256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505817e5-e790-460e-aa44-957eccfc3a10 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.862686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-505817e5-e790-460e-aa44-957eccfc3a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:06.928163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505817e5-e790-460e-aa44-957eccfc3a10 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:06.935267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e87883-b1f8-4c28-a887-54c94f24c7c1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:06.942963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0092fa-8549-4dd3-b969-72efd2bcf5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:06.943765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0092fa-8549-4dd3-b969-72efd2bcf5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:06.944179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0092fa-8549-4dd3-b969-72efd2bcf5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:06.944844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0092fa-8549-4dd3-b969-72efd2bcf5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:06.945878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0092fa-8549-4dd3-b969-72efd2bcf5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:06.951251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:06.953424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0092fa-8549-4dd3-b969-72efd2bcf5d1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:06.974663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1133/2256] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:06.981345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0092fa-8549-4dd3-b969-72efd2bcf5d1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:06.994049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0092fa-8549-4dd3-b969-72efd2bcf5d1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:07.002029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30479a68-24e2-400a-b761-5e3d767a1922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:07.003048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479a68-24e2-400a-b761-5e3d767a1922 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:07.003486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479a68-24e2-400a-b761-5e3d767a1922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:07.003980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479a68-24e2-400a-b761-5e3d767a1922 admin admin] PATH_INFO: `/v3/projects/7f855614e08944aa8c780366cfe3fd17/users/9c5af3de8f1b4671a023bc84594b7dcc/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:07.007271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30479a68-24e2-400a-b761-5e3d767a1922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f855614e08944aa8c780366cfe3fd17, user_id=9c5af3de8f1b4671a023bc84594b7dcc, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:07.012078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30479a68-24e2-400a-b761-5e3d767a1922 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:07.019962 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b915c-209c-4494-bb4c-2cc0d8ae8413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:07.020915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b915c-209c-4494-bb4c-2cc0d8ae8413 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:07.021170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b915c-209c-4494-bb4c-2cc0d8ae8413 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:07.021449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b915c-209c-4494-bb4c-2cc0d8ae8413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:07.036084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1134/2257] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:06 2022] PUT /identity/v3/projects/7f855614e08944aa8c780366cfe3fd17/users/9c5af3de8f1b4671a023bc84594b7dcc/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:07.037615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc551f9-1551-4e27-836c-6c66a5c273f2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:07.037910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc551f9-1551-4e27-836c-6c66a5c273f2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:07.038285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc551f9-1551-4e27-836c-6c66a5c273f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:07.077463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2bc551f9-1551-4e27-836c-6c66a5c273f2 None None] MFA Rules not processed for user `9c5af3de8f1b4671a023bc84594b7dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:07.078882 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2bc551f9-1551-4e27-836c-6c66a5c273f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:07.090035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b915c-209c-4494-bb4c-2cc0d8ae8413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:07.091561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b915c-209c-4494-bb4c-2cc0d8ae8413 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:07.106596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1124/2258] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:24:07.186270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1135/2259] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:07.197911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b915c-209c-4494-bb4c-2cc0d8ae8413 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:07.199420 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b15b915c-209c-4494-bb4c-2cc0d8ae8413 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:07.207722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:07.212497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b915c-209c-4494-bb4c-2cc0d8ae8413 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:07.271370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07873b-c870-4c78-bd3f-68759f2cc7d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:07.272063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07873b-c870-4c78-bd3f-68759f2cc7d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:07.274409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07873b-c870-4c78-bd3f-68759f2cc7d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:07.274409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07873b-c870-4c78-bd3f-68759f2cc7d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:07.339515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07873b-c870-4c78-bd3f-68759f2cc7d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:07.341082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07873b-c870-4c78-bd3f-68759f2cc7d2 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:07.352097 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1125/2260] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:07.522277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:08.614913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:08.659011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:09.370557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe60b9-35a3-4df8-9714-3576e4027aca None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:09.370835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe60b9-35a3-4df8-9714-3576e4027aca None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:09.371110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe60b9-35a3-4df8-9714-3576e4027aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:09.391737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5bfe60b9-35a3-4df8-9714-3576e4027aca None None] MFA Rules not processed for user `b07c458f1a5c4391aaaa79af49a236c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:09.392955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5bfe60b9-35a3-4df8-9714-3576e4027aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:09.452278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1136/2261] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:13.666386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07873b-c870-4c78-bd3f-68759f2cc7d2 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:13.759594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07873b-c870-4c78-bd3f-68759f2cc7d2 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:13.765890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:13.766597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:13.767045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:13.767526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] PATH_INFO: `/v3/users/1584f19f761549dc9b40ead80ea14224` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:13.769344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1584f19f761549dc9b40ead80ea14224)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:13.771898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:14.021708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1584f19f761549dc9b40ead80ea14224'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:14.030729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1584f19f761549dc9b40ead80ea14224'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:14.111226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1584f19f761549dc9b40ead80ea14224'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:14.112388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1584f19f761549dc9b40ead80ea14224'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:14.114222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1584f19f761549dc9b40ead80ea14224'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:14.139041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1126/2262] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:13 2022] DELETE /identity/v3/users/1584f19f761549dc9b40ead80ea14224 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:14.141229 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a2e3e-c951-4a31-b974-1453b7dc06a1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:14.142061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a2e3e-c951-4a31-b974-1453b7dc06a1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:14.142515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a2e3e-c951-4a31-b974-1453b7dc06a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:14.163809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5b3a2e3e-c951-4a31-b974-1453b7dc06a1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:14.165324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5b3a2e3e-c951-4a31-b974-1453b7dc06a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:14.268659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1137/2263] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:14.275264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:14.277036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:14.344707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b60eb-f229-4d66-9cd3-d64477568e04 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:14.348436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc95a4-4788-47c7-9126-7995eae2a8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:14.349063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc95a4-4788-47c7-9126-7995eae2a8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:14.349345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc95a4-4788-47c7-9126-7995eae2a8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:14.349622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc95a4-4788-47c7-9126-7995eae2a8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:14.359933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc95a4-4788-47c7-9126-7995eae2a8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:14.361461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc95a4-4788-47c7-9126-7995eae2a8a0 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:14.370931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1127/2264] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:24:14.820944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a2e3e-c951-4a31-b974-1453b7dc06a1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:14.822644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5b3a2e3e-c951-4a31-b974-1453b7dc06a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:14.836744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a2e3e-c951-4a31-b974-1453b7dc06a1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:14.844154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:14.845011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:14.856818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:14.864536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] PATH_INFO: `/v3/projects/a3615c2b50b341508e01f54bf81e6eb6` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:14.864536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3615c2b50b341508e01f54bf81e6eb6)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:14.866326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:14.907352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:15.041984 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3615c2b50b341508e01f54bf81e6eb6', 'request_id': 'req-59e823db-0cbd-41d0-8b3f-a003dc558172'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:15.067964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3615c2b50b341508e01f54bf81e6eb6', 'request_id': 'req-59e823db-0cbd-41d0-8b3f-a003dc558172'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:15.087811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3615c2b50b341508e01f54bf81e6eb6', 'request_id': 'req-59e823db-0cbd-41d0-8b3f-a003dc558172'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:15.167715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] The token cache is being invalidate because project a3615c2b50b341508e01f54bf81e6eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:15.169019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:15.173317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-59e823db-0cbd-41d0-8b3f-a003dc558172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:15.176374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1138/2265] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:14 2022] DELETE /identity/v3/projects/a3615c2b50b341508e01f54bf81e6eb6 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:15.182608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03f619-dc86-43ba-a546-71d5741e727d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:15.183096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03f619-dc86-43ba-a546-71d5741e727d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:15.183096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03f619-dc86-43ba-a546-71d5741e727d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:15.207342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2a03f619-dc86-43ba-a546-71d5741e727d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:15.207342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2a03f619-dc86-43ba-a546-71d5741e727d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:15.241854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:15.308866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1128/2266] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:15.318400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8de39-2e22-40e4-b509-73617caff1df None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:15.318896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8de39-2e22-40e4-b509-73617caff1df None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:15.319333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8de39-2e22-40e4-b509-73617caff1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:15.361867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-56e8de39-2e22-40e4-b509-73617caff1df None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:15.363337 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-56e8de39-2e22-40e4-b509-73617caff1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:15.372733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1139/2267] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:15.378393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03f619-dc86-43ba-a546-71d5741e727d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:15.399483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03f619-dc86-43ba-a546-71d5741e727d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:15.406581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e17005-bbf9-4b92-9011-f1b512367766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:15.420874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e17005-bbf9-4b92-9011-f1b512367766 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:15.420874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e17005-bbf9-4b92-9011-f1b512367766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:15.420874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e17005-bbf9-4b92-9011-f1b512367766 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:15.420874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e17005-bbf9-4b92-9011-f1b512367766 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:15.420874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e17005-bbf9-4b92-9011-f1b512367766 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:15.420874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e17005-bbf9-4b92-9011-f1b512367766 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:15.426221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1129/2268] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:24:15.430362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8de39-2e22-40e4-b509-73617caff1df None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:15.444599 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8de39-2e22-40e4-b509-73617caff1df None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:15.450542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30486c-348c-4275-8787-d392eead96db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:15.451247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30486c-348c-4275-8787-d392eead96db admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:15.451607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30486c-348c-4275-8787-d392eead96db admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:15.451977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30486c-348c-4275-8787-d392eead96db admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:15.453265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30486c-348c-4275-8787-d392eead96db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:15.455246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30486c-348c-4275-8787-d392eead96db admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:15.591592 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1140/2269] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:15 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:15.596622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e17005-bbf9-4b92-9011-f1b512367766 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:15.609131 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e17005-bbf9-4b92-9011-f1b512367766 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:15.702116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f510785-16e2-488a-ae01-a3d33e3de1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:15.704123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f510785-16e2-488a-ae01-a3d33e3de1e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:15.708179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f510785-16e2-488a-ae01-a3d33e3de1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:15.708467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f510785-16e2-488a-ae01-a3d33e3de1e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:15.709192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f510785-16e2-488a-ae01-a3d33e3de1e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:15.711169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f510785-16e2-488a-ae01-a3d33e3de1e3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:15.825973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1130/2270] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:15 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:15.830559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30486c-348c-4275-8787-d392eead96db admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:15.844001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30486c-348c-4275-8787-d392eead96db admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:15.849074 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb9ec9-9183-42e7-858c-8d7cb5cebf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:15.850038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb9ec9-9183-42e7-858c-8d7cb5cebf18 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:15.850486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb9ec9-9183-42e7-858c-8d7cb5cebf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:15.850908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb9ec9-9183-42e7-858c-8d7cb5cebf18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:15.851867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb9ec9-9183-42e7-858c-8d7cb5cebf18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:15.853682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb9ec9-9183-42e7-858c-8d7cb5cebf18 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:15.866398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1141/2271] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:15.870532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f510785-16e2-488a-ae01-a3d33e3de1e3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:15.884561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f510785-16e2-488a-ae01-a3d33e3de1e3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:15.891489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8bd01-2ed0-4fef-bbb0-dec25868efba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:15.892206 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8bd01-2ed0-4fef-bbb0-dec25868efba admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:15.892543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8bd01-2ed0-4fef-bbb0-dec25868efba admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:15.892851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8bd01-2ed0-4fef-bbb0-dec25868efba admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:15.893583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8bd01-2ed0-4fef-bbb0-dec25868efba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:15.895200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8bd01-2ed0-4fef-bbb0-dec25868efba admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:15.910364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1131/2272] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:24:15.915605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb9ec9-9183-42e7-858c-8d7cb5cebf18 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:15.924354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb9ec9-9183-42e7-858c-8d7cb5cebf18 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:15.931829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53df06-15d1-4c81-8f70-981d74f70aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:15.932537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53df06-15d1-4c81-8f70-981d74f70aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:15.932853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53df06-15d1-4c81-8f70-981d74f70aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:15.933115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53df06-15d1-4c81-8f70-981d74f70aa5 admin admin] PATH_INFO: `/v3/projects/1625d1429d294a2193fb22b2104384bf/users/aa886e3c6aac40c8975bad310f65e517/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:15.958505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53df06-15d1-4c81-8f70-981d74f70aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1625d1429d294a2193fb22b2104384bf, user_id=aa886e3c6aac40c8975bad310f65e517, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:15.961552 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53df06-15d1-4c81-8f70-981d74f70aa5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:16.076766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1142/2273] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:15 2022] PUT /identity/v3/projects/1625d1429d294a2193fb22b2104384bf/users/aa886e3c6aac40c8975bad310f65e517/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:16.082311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8848a7f-5b82-4c15-8b32-3ceb7cdcd00f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:16.082790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8848a7f-5b82-4c15-8b32-3ceb7cdcd00f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:16.083203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8848a7f-5b82-4c15-8b32-3ceb7cdcd00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:16.113481 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f8848a7f-5b82-4c15-8b32-3ceb7cdcd00f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:16.115149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f8848a7f-5b82-4c15-8b32-3ceb7cdcd00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:16.201765 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1132/2274] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:16.212291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53df06-15d1-4c81-8f70-981d74f70aa5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:16.214255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ee53df06-15d1-4c81-8f70-981d74f70aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:16.294210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53df06-15d1-4c81-8f70-981d74f70aa5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:16.301070 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df265ba-60ff-43bc-a63b-6f03c81842cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:16.301659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df265ba-60ff-43bc-a63b-6f03c81842cf service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:16.302011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df265ba-60ff-43bc-a63b-6f03c81842cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:16.302353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df265ba-60ff-43bc-a63b-6f03c81842cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:16.324014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df265ba-60ff-43bc-a63b-6f03c81842cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:16.324014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df265ba-60ff-43bc-a63b-6f03c81842cf service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:16.330987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1143/2275] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:16.342716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8848a7f-5b82-4c15-8b32-3ceb7cdcd00f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:16.344971 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f8848a7f-5b82-4c15-8b32-3ceb7cdcd00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:16.373762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:16.403882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8848a7f-5b82-4c15-8b32-3ceb7cdcd00f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:16.413401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17739dc-d77d-4102-8972-2319dceeca9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:16.414045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17739dc-d77d-4102-8972-2319dceeca9d service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:16.414296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17739dc-d77d-4102-8972-2319dceeca9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:16.414628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17739dc-d77d-4102-8972-2319dceeca9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:16.417332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b17739dc-d77d-4102-8972-2319dceeca9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:16.510682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17739dc-d77d-4102-8972-2319dceeca9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:16.512181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17739dc-d77d-4102-8972-2319dceeca9d service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:16.522580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1133/2276] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:22.175646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf999b63-3751-4921-8148-ca9dbe878d33 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:22.176774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf999b63-3751-4921-8148-ca9dbe878d33 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:22.177058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf999b63-3751-4921-8148-ca9dbe878d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:22.219932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-cf999b63-3751-4921-8148-ca9dbe878d33 None None] MFA Rules not processed for user `aa886e3c6aac40c8975bad310f65e517`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:22.221551 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-cf999b63-3751-4921-8148-ca9dbe878d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:22.277762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:22.310439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1144/2277] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:22.315624 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17739dc-d77d-4102-8972-2319dceeca9d service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:22.329879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17739dc-d77d-4102-8972-2319dceeca9d service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:22.334497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a8893-7027-4010-b89a-178c000fad33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:22.335146 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a8893-7027-4010-b89a-178c000fad33 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:22.335427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a8893-7027-4010-b89a-178c000fad33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:22.335701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a8893-7027-4010-b89a-178c000fad33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:22.336413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a8893-7027-4010-b89a-178c000fad33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:22.339087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a8893-7027-4010-b89a-178c000fad33 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:22.525357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1134/2278] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:22 2022] POST /identity/v3/projects => generated 373 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:22.526373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf999b63-3751-4921-8148-ca9dbe878d33 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:22.554634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf999b63-3751-4921-8148-ca9dbe878d33 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:22.645115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a088304-4349-4043-b867-5aa42b0a05fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:22.646063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a088304-4349-4043-b867-5aa42b0a05fd admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:22.646488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a088304-4349-4043-b867-5aa42b0a05fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:22.646799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a088304-4349-4043-b867-5aa42b0a05fd admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:22.647713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a088304-4349-4043-b867-5aa42b0a05fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:22.653314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a088304-4349-4043-b867-5aa42b0a05fd admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:22.759910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1145/2279] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:22 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:22.774537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a8893-7027-4010-b89a-178c000fad33 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:22.788568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a8893-7027-4010-b89a-178c000fad33 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:22.794105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635dc96-e29f-4a1a-8d16-2a3515f85d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:22.795047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635dc96-e29f-4a1a-8d16-2a3515f85d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:22.804000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635dc96-e29f-4a1a-8d16-2a3515f85d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:22.804000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635dc96-e29f-4a1a-8d16-2a3515f85d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:22.804000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635dc96-e29f-4a1a-8d16-2a3515f85d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:22.804000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635dc96-e29f-4a1a-8d16-2a3515f85d45 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:22.816450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1135/2280] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:22.818233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a088304-4349-4043-b867-5aa42b0a05fd admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:22.833935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a088304-4349-4043-b867-5aa42b0a05fd admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:22.841235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a55b01-d009-437c-a434-abac44516482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:22.842405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a55b01-d009-437c-a434-abac44516482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:22.842917 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a55b01-d009-437c-a434-abac44516482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:22.844346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a55b01-d009-437c-a434-abac44516482 admin admin] PATH_INFO: `/v3/projects/dc146b11964e4d6f9833033f987ff310/users/d327cdb7279d487289c113b25bf59426/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:22.864051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a55b01-d009-437c-a434-abac44516482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc146b11964e4d6f9833033f987ff310, user_id=d327cdb7279d487289c113b25bf59426, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:22.867432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a55b01-d009-437c-a434-abac44516482 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:23.006841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635dc96-e29f-4a1a-8d16-2a3515f85d45 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:23.007373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1146/2281] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:22 2022] PUT /identity/v3/projects/dc146b11964e4d6f9833033f987ff310/users/d327cdb7279d487289c113b25bf59426/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:23.021349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635dc96-e29f-4a1a-8d16-2a3515f85d45 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:23.170631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d1de3-be4f-4aff-a3a2-f2c1d55cff7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:23.170631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d1de3-be4f-4aff-a3a2-f2c1d55cff7c admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:23.170631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d1de3-be4f-4aff-a3a2-f2c1d55cff7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:23.170631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d1de3-be4f-4aff-a3a2-f2c1d55cff7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:23.171088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d1de3-be4f-4aff-a3a2-f2c1d55cff7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:23.171824 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d1de3-be4f-4aff-a3a2-f2c1d55cff7c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:23.182781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1136/2282] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:23.187075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a55b01-d009-437c-a434-abac44516482 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:23.201056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a55b01-d009-437c-a434-abac44516482 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:23.206540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53f108-eec2-4262-8aef-97404e71bac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:23.207224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53f108-eec2-4262-8aef-97404e71bac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:23.207509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53f108-eec2-4262-8aef-97404e71bac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:23.207812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53f108-eec2-4262-8aef-97404e71bac6 admin admin] PATH_INFO: `/v3/projects/dc146b11964e4d6f9833033f987ff310/users/d327cdb7279d487289c113b25bf59426/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:23.209795 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53f108-eec2-4262-8aef-97404e71bac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc146b11964e4d6f9833033f987ff310, user_id=d327cdb7279d487289c113b25bf59426, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:23.212612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53f108-eec2-4262-8aef-97404e71bac6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:23.226866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1147/2283] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:23 2022] PUT /identity/v3/projects/dc146b11964e4d6f9833033f987ff310/users/d327cdb7279d487289c113b25bf59426/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:24.939205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d1de3-be4f-4aff-a3a2-f2c1d55cff7c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:24.941074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f27d1de3-be4f-4aff-a3a2-f2c1d55cff7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:24.963541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d1de3-be4f-4aff-a3a2-f2c1d55cff7c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:25.028370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:25.068827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:25.074831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:25.074831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:25.074831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] PATH_INFO: `/v3/users/ce5998047c5b449c9573e6f4b5ade27e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:25.074831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5998047c5b449c9573e6f4b5ade27e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:25.074831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:25.318753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5998047c5b449c9573e6f4b5ade27e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:25.327470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5998047c5b449c9573e6f4b5ade27e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:25.412118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5998047c5b449c9573e6f4b5ade27e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:25.412966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5998047c5b449c9573e6f4b5ade27e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:25.413530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5998047c5b449c9573e6f4b5ade27e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:25.482556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1137/2284] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:24 2022] DELETE /identity/v3/users/ce5998047c5b449c9573e6f4b5ade27e => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:25.487864 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38f22d-4253-40e5-b27a-795920773d7e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:25.488356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38f22d-4253-40e5-b27a-795920773d7e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:25.488673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38f22d-4253-40e5-b27a-795920773d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:25.589188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1a38f22d-4253-40e5-b27a-795920773d7e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:25.590468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1a38f22d-4253-40e5-b27a-795920773d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:25.711664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1148/2285] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:25.752496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:25.755076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:25.840593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f41848-185b-4a95-b38e-f6cd633343c0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:25.857896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232ee45-1371-45b4-b2b8-66a51d7a6e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:25.859534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232ee45-1371-45b4-b2b8-66a51d7a6e0f service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:25.860076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232ee45-1371-45b4-b2b8-66a51d7a6e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:25.860697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232ee45-1371-45b4-b2b8-66a51d7a6e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:25.871172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232ee45-1371-45b4-b2b8-66a51d7a6e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:25.873215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232ee45-1371-45b4-b2b8-66a51d7a6e0f service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:25.885824 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1138/2286] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:26.904307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38f22d-4253-40e5-b27a-795920773d7e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:26.906184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1a38f22d-4253-40e5-b27a-795920773d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:26.929255 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38f22d-4253-40e5-b27a-795920773d7e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:26.939897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:26.941228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:26.941594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:26.941951 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] PATH_INFO: `/v3/users/0f5eeb67fbe84d38827cc4520ac1b423` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:26.945570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5eeb67fbe84d38827cc4520ac1b423)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:26.949301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:27.130627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:27.137356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232ee45-1371-45b4-b2b8-66a51d7a6e0f service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:27.138885 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9232ee45-1371-45b4-b2b8-66a51d7a6e0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:27.165609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232ee45-1371-45b4-b2b8-66a51d7a6e0f service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:27.178024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:27.178951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:27.179499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:27.179997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] PATH_INFO: `/v3/projects/4bd65d186e3f41cc9541660aee8b1e52` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:27.182631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd65d186e3f41cc9541660aee8b1e52)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:27.186430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:27.241248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5eeb67fbe84d38827cc4520ac1b423'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:27.260842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5eeb67fbe84d38827cc4520ac1b423'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:27.345578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5eeb67fbe84d38827cc4520ac1b423'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:27.346656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5eeb67fbe84d38827cc4520ac1b423'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:27.369978 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd65d186e3f41cc9541660aee8b1e52', 'request_id': 'req-5e0dad88-f54b-4ade-b79e-802b780cf582'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:27.372464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-f84c2f2a-28be-4ff9-8e32-fd8126fcf20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5eeb67fbe84d38827cc4520ac1b423'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:27.383270 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:27.398722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd65d186e3f41cc9541660aee8b1e52', 'request_id': 'req-5e0dad88-f54b-4ade-b79e-802b780cf582'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:27.405328 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1149/2287] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:26 2022] DELETE /identity/v3/users/0f5eeb67fbe84d38827cc4520ac1b423 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:27.408947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8135628-3435-4ed5-a5b5-36ad7110a1fb None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:27.409753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8135628-3435-4ed5-a5b5-36ad7110a1fb None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:27.410064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8135628-3435-4ed5-a5b5-36ad7110a1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:27.453482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c8135628-3435-4ed5-a5b5-36ad7110a1fb None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:27.453482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c8135628-3435-4ed5-a5b5-36ad7110a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:27.537849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd65d186e3f41cc9541660aee8b1e52', 'request_id': 'req-5e0dad88-f54b-4ade-b79e-802b780cf582'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:27.557178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] The token cache is being invalidate because project 4bd65d186e3f41cc9541660aee8b1e52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:27.557450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:27.560370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:27.564158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1139/2288] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:27 2022] DELETE /identity/v3/projects/4bd65d186e3f41cc9541660aee8b1e52 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:27.567980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:27.569865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:27.617830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1150/2289] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:27.677370 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8135628-3435-4ed5-a5b5-36ad7110a1fb None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:27.679690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c8135628-3435-4ed5-a5b5-36ad7110a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:27.722021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dad88-f54b-4ade-b79e-802b780cf582 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:27.732560 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:27.733167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:27.733450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:27.733724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] PATH_INFO: `/v3/users/9c5af3de8f1b4671a023bc84594b7dcc` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:27.740315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c5af3de8f1b4671a023bc84594b7dcc)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:27.746214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:27.804167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8135628-3435-4ed5-a5b5-36ad7110a1fb None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:27.809170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f0beb-c6d5-425b-b0f0-8a45403b9288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:27.809822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f0beb-c6d5-425b-b0f0-8a45403b9288 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:27.810902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f0beb-c6d5-425b-b0f0-8a45403b9288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:27.810902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f0beb-c6d5-425b-b0f0-8a45403b9288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:27.834160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f0beb-c6d5-425b-b0f0-8a45403b9288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:27.836071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f0beb-c6d5-425b-b0f0-8a45403b9288 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:27.860517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1151/2290] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:24:28.028687 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c5af3de8f1b4671a023bc84594b7dcc'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:28.033638 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c5af3de8f1b4671a023bc84594b7dcc'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:28.127147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c5af3de8f1b4671a023bc84594b7dcc'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:28.127960 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c5af3de8f1b4671a023bc84594b7dcc'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:28.128680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-e3049c42-b0c1-4630-aaca-edee54c7788d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c5af3de8f1b4671a023bc84594b7dcc'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:28.163507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1140/2291] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:27 2022] DELETE /identity/v3/users/9c5af3de8f1b4671a023bc84594b7dcc => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:28.760429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f0beb-c6d5-425b-b0f0-8a45403b9288 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:28.775712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f0beb-c6d5-425b-b0f0-8a45403b9288 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:28.904949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:28.920744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:28.922203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:28.922664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:28.923538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] PATH_INFO: `/v3/projects/7f855614e08944aa8c780366cfe3fd17` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:28.935707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f855614e08944aa8c780366cfe3fd17)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:28.941222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:29.083737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f855614e08944aa8c780366cfe3fd17', 'request_id': 'req-88d7d20f-010f-458c-a4ac-4e050cf732a1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:29.094646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f855614e08944aa8c780366cfe3fd17', 'request_id': 'req-88d7d20f-010f-458c-a4ac-4e050cf732a1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:29.100863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f855614e08944aa8c780366cfe3fd17', 'request_id': 'req-88d7d20f-010f-458c-a4ac-4e050cf732a1'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:29.202125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] The token cache is being invalidate because project 7f855614e08944aa8c780366cfe3fd17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:29.204351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:29.205842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-88d7d20f-010f-458c-a4ac-4e050cf732a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:29.213207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1152/2292] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:28 2022] DELETE /identity/v3/projects/7f855614e08944aa8c780366cfe3fd17 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:29.231400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356b231-3f63-493b-9149-ec92699d475d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:29.232265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356b231-3f63-493b-9149-ec92699d475d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:29.232265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356b231-3f63-493b-9149-ec92699d475d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:29.254263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2356b231-3f63-493b-9149-ec92699d475d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:29.255507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2356b231-3f63-493b-9149-ec92699d475d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:29.345064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0f924-bc07-4758-9c35-1bde8fb23af4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:29.345386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0f924-bc07-4758-9c35-1bde8fb23af4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:29.345631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0f924-bc07-4758-9c35-1bde8fb23af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:29.348393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1141/2293] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:29.375212 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-46a0f924-bc07-4758-9c35-1bde8fb23af4 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:29.377049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-46a0f924-bc07-4758-9c35-1bde8fb23af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:29.392065 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1153/2294] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:29.397049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356b231-3f63-493b-9149-ec92699d475d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:29.411671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:29.419522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356b231-3f63-493b-9149-ec92699d475d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:29.427987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cb971-fae9-47c8-892e-51a2c237c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:29.428995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cb971-fae9-47c8-892e-51a2c237c6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:29.429831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cb971-fae9-47c8-892e-51a2c237c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:29.430433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cb971-fae9-47c8-892e-51a2c237c6d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:29.432228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cb971-fae9-47c8-892e-51a2c237c6d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:29.433534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cb971-fae9-47c8-892e-51a2c237c6d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:29.436319 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cb971-fae9-47c8-892e-51a2c237c6d1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:29.468725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1142/2295] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:24:29.473142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0f924-bc07-4758-9c35-1bde8fb23af4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:29.483446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0f924-bc07-4758-9c35-1bde8fb23af4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:29.488608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ca2ef-5f57-4b29-815b-b19d55fdd30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:29.489446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ca2ef-5f57-4b29-815b-b19d55fdd30a admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:29.497363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ca2ef-5f57-4b29-815b-b19d55fdd30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:29.497576 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ca2ef-5f57-4b29-815b-b19d55fdd30a admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:29.498295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ca2ef-5f57-4b29-815b-b19d55fdd30a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:29.501646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ca2ef-5f57-4b29-815b-b19d55fdd30a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:29.503390 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cb971-fae9-47c8-892e-51a2c237c6d1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:29.505457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f17cb971-fae9-47c8-892e-51a2c237c6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:29.532812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cb971-fae9-47c8-892e-51a2c237c6d1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:29.555115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:29.559004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:29.563870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:29.563870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] PATH_INFO: `/v3/projects/cf92482579224b5abbbe96c35210f19f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:29.564578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf92482579224b5abbbe96c35210f19f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:29.567191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:29.700373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1154/2296] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:29 2022] POST /identity/v3/projects => generated 375 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:24:29.701878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ca2ef-5f57-4b29-815b-b19d55fdd30a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:29.736804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf92482579224b5abbbe96c35210f19f', 'request_id': 'req-f872abc9-8c69-4065-bfde-4c8007ad7394'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:29.749613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf92482579224b5abbbe96c35210f19f', 'request_id': 'req-f872abc9-8c69-4065-bfde-4c8007ad7394'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:29.818508 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf92482579224b5abbbe96c35210f19f', 'request_id': 'req-f872abc9-8c69-4065-bfde-4c8007ad7394'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:29.837225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] The token cache is being invalidate because project cf92482579224b5abbbe96c35210f19f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:29.837525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:29.842615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ca2ef-5f57-4b29-815b-b19d55fdd30a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:29.848303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f872abc9-8c69-4065-bfde-4c8007ad7394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:29.849995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1143/2297] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:29 2022] DELETE /identity/v3/projects/cf92482579224b5abbbe96c35210f19f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:24:29.855078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da0ba3-ce49-49c2-8579-ab073221f810 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:29.855588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da0ba3-ce49-49c2-8579-ab073221f810 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:29.855891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da0ba3-ce49-49c2-8579-ab073221f810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:29.892806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d8da0ba3-ce49-49c2-8579-ab073221f810 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:29.894121 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d8da0ba3-ce49-49c2-8579-ab073221f810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:29.949463 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc579c3d-bce4-4fa7-b141-1e09c2229292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:29.953287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579c3d-bce4-4fa7-b141-1e09c2229292 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:29.953287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579c3d-bce4-4fa7-b141-1e09c2229292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:29.953287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579c3d-bce4-4fa7-b141-1e09c2229292 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:29.954896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc579c3d-bce4-4fa7-b141-1e09c2229292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:29.957303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc579c3d-bce4-4fa7-b141-1e09c2229292 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:30.008233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1144/2298] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:30.018903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5a62-084f-4d59-b37f-bcf32a7918a3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.026633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5a62-084f-4d59-b37f-bcf32a7918a3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.028227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5a62-084f-4d59-b37f-bcf32a7918a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.063939 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-732b5a62-084f-4d59-b37f-bcf32a7918a3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:30.065674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-732b5a62-084f-4d59-b37f-bcf32a7918a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:30.075997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1145/2299] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:30.081245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b5a62-084f-4d59-b37f-bcf32a7918a3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:30.096165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b5a62-084f-4d59-b37f-bcf32a7918a3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:30.107373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340ab50-2eb2-4dfb-afe5-2ea753792d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:30.108191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340ab50-2eb2-4dfb-afe5-2ea753792d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.108544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340ab50-2eb2-4dfb-afe5-2ea753792d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.108912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340ab50-2eb2-4dfb-afe5-2ea753792d53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.109720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340ab50-2eb2-4dfb-afe5-2ea753792d53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:30.110037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340ab50-2eb2-4dfb-afe5-2ea753792d53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:30.111857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340ab50-2eb2-4dfb-afe5-2ea753792d53 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:30.128796 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1146/2300] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:24:30.129917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340ab50-2eb2-4dfb-afe5-2ea753792d53 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:30.140028 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340ab50-2eb2-4dfb-afe5-2ea753792d53 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:30.154481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1155/2301] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:29 2022] POST /identity/v3/users => generated 288 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:30.158806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc579c3d-bce4-4fa7-b141-1e09c2229292 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:30.162270 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bc579c3d-bce4-4fa7-b141-1e09c2229292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:30.166058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac05887-84f7-409f-aacb-c21c0059afab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:30.168134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac05887-84f7-409f-aacb-c21c0059afab admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.168417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac05887-84f7-409f-aacb-c21c0059afab admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.171806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac05887-84f7-409f-aacb-c21c0059afab admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.172520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac05887-84f7-409f-aacb-c21c0059afab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:30.174473 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac05887-84f7-409f-aacb-c21c0059afab admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:30.186303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc579c3d-bce4-4fa7-b141-1e09c2229292 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:30.197756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fc362-0834-4327-8524-56abf4941c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:30.198394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fc362-0834-4327-8524-56abf4941c52 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.198678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fc362-0834-4327-8524-56abf4941c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.198955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fc362-0834-4327-8524-56abf4941c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.199701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fc362-0834-4327-8524-56abf4941c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:30.201347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fc362-0834-4327-8524-56abf4941c52 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:30.219380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1156/2302] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:30.220472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fc362-0834-4327-8524-56abf4941c52 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:30.231607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fc362-0834-4327-8524-56abf4941c52 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:30.244297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d54fa-b90e-4f6e-9aed-009df572c451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:30.244942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d54fa-b90e-4f6e-9aed-009df572c451 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.245225 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d54fa-b90e-4f6e-9aed-009df572c451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.245502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d54fa-b90e-4f6e-9aed-009df572c451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.246218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d54fa-b90e-4f6e-9aed-009df572c451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:30.248405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d54fa-b90e-4f6e-9aed-009df572c451 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:30.253502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:30.300344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1157/2303] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:30.301639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d54fa-b90e-4f6e-9aed-009df572c451 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:30.305257 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1147/2304] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:30 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:30.310610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac05887-84f7-409f-aacb-c21c0059afab admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:30.328793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d54fa-b90e-4f6e-9aed-009df572c451 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:30.422506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac05887-84f7-409f-aacb-c21c0059afab admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:30.427525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bda39-6051-466f-8c40-6825c15f030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:30.428428 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bda39-6051-466f-8c40-6825c15f030b admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.428935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bda39-6051-466f-8c40-6825c15f030b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.429453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bda39-6051-466f-8c40-6825c15f030b admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.430311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bda39-6051-466f-8c40-6825c15f030b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:30.432483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bda39-6051-466f-8c40-6825c15f030b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:30.483099 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92dca5-7716-4dab-8bad-7014f432fa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:30.483811 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92dca5-7716-4dab-8bad-7014f432fa8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.484670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92dca5-7716-4dab-8bad-7014f432fa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.485180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92dca5-7716-4dab-8bad-7014f432fa8e admin admin] PATH_INFO: `/v3/projects/999498337156490dbbe38504a38c8035/users/9a6dbaea48f547e99b40c7acd114489e/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.505179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92dca5-7716-4dab-8bad-7014f432fa8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=999498337156490dbbe38504a38c8035, user_id=9a6dbaea48f547e99b40c7acd114489e, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:30.507890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92dca5-7716-4dab-8bad-7014f432fa8e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:30.557764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1148/2305] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:30 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:30.561591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bda39-6051-466f-8c40-6825c15f030b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:30.575352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bda39-6051-466f-8c40-6825c15f030b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:30.580730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03843c-beb8-40e3-9852-7d6ff040e7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:30.581548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03843c-beb8-40e3-9852-7d6ff040e7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.581979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03843c-beb8-40e3-9852-7d6ff040e7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.582402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03843c-beb8-40e3-9852-7d6ff040e7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.583450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03843c-beb8-40e3-9852-7d6ff040e7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:30.585505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03843c-beb8-40e3-9852-7d6ff040e7b5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:30.599250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1149/2306] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:24:30.607556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03843c-beb8-40e3-9852-7d6ff040e7b5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:30.624055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1158/2307] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:30 2022] PUT /identity/v3/projects/999498337156490dbbe38504a38c8035/users/9a6dbaea48f547e99b40c7acd114489e/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:30.631994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03843c-beb8-40e3-9852-7d6ff040e7b5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:30.636354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577223f0-e42e-44ce-a940-4473b0782904 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.640086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577223f0-e42e-44ce-a940-4473b0782904 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.640434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577223f0-e42e-44ce-a940-4473b0782904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.699226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-577223f0-e42e-44ce-a940-4473b0782904 None None] MFA Rules not processed for user `9a6dbaea48f547e99b40c7acd114489e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:30.700516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-577223f0-e42e-44ce-a940-4473b0782904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:30.739355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb09a38-d5b4-4dad-a76e-806fa0179646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:30.740056 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb09a38-d5b4-4dad-a76e-806fa0179646 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.740411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb09a38-d5b4-4dad-a76e-806fa0179646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.740749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb09a38-d5b4-4dad-a76e-806fa0179646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.741554 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb09a38-d5b4-4dad-a76e-806fa0179646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:30.743398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb09a38-d5b4-4dad-a76e-806fa0179646 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:30.755661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1150/2308] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:30.759477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb09a38-d5b4-4dad-a76e-806fa0179646 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:30.785331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb09a38-d5b4-4dad-a76e-806fa0179646 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:30.791764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b01ca9-d240-4036-9ca4-9473c9e9f0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:30.792688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b01ca9-d240-4036-9ca4-9473c9e9f0a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.796017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b01ca9-d240-4036-9ca4-9473c9e9f0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.796017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b01ca9-d240-4036-9ca4-9473c9e9f0a2 admin admin] PATH_INFO: `/v3/projects/aeee08ad0a654fc1839cc533fc804479/users/42047fe93eb24adbbc24844a75ed21f2/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.816149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b01ca9-d240-4036-9ca4-9473c9e9f0a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeee08ad0a654fc1839cc533fc804479, user_id=42047fe93eb24adbbc24844a75ed21f2, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:30.819950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b01ca9-d240-4036-9ca4-9473c9e9f0a2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:30.835897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1159/2309] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:24:30.840872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db52b0-292a-4d18-9848-e3a049ec96b7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:30.840872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db52b0-292a-4d18-9848-e3a049ec96b7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:30.840872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db52b0-292a-4d18-9848-e3a049ec96b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:30.882970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-91db52b0-292a-4d18-9848-e3a049ec96b7 None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:30.885436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-91db52b0-292a-4d18-9848-e3a049ec96b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:30.955962 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1151/2310] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:30 2022] PUT /identity/v3/projects/aeee08ad0a654fc1839cc533fc804479/users/42047fe93eb24adbbc24844a75ed21f2/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:30.955962 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b01ca9-d240-4036-9ca4-9473c9e9f0a2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:31.001836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1160/2311] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:24:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:31.003541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a70cb3-39e3-4157-90fd-c8b9177f0efc None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:31.004095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a70cb3-39e3-4157-90fd-c8b9177f0efc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:31.004602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a70cb3-39e3-4157-90fd-c8b9177f0efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:31.056879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-c0a70cb3-39e3-4157-90fd-c8b9177f0efc None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:31.059532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c0a70cb3-39e3-4157-90fd-c8b9177f0efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:31.085583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b01ca9-d240-4036-9ca4-9473c9e9f0a2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:31.093997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:31.100798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51d3dc-0a2d-4fa2-9da8-2f1f5231d6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:31.101453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51d3dc-0a2d-4fa2-9da8-2f1f5231d6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:31.101912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51d3dc-0a2d-4fa2-9da8-2f1f5231d6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:31.102185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51d3dc-0a2d-4fa2-9da8-2f1f5231d6d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:31.102869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51d3dc-0a2d-4fa2-9da8-2f1f5231d6d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:31.107880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51d3dc-0a2d-4fa2-9da8-2f1f5231d6d0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:31.186792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1161/2312] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:31.186792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a70cb3-39e3-4157-90fd-c8b9177f0efc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:31.186792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c0a70cb3-39e3-4157-90fd-c8b9177f0efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:31.233012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1152/2313] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:30 2022] POST /identity/v3/projects => generated 375 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:31.234101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51d3dc-0a2d-4fa2-9da8-2f1f5231d6d0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:31.236497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ad51d3dc-0a2d-4fa2-9da8-2f1f5231d6d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:31.325542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a70cb3-39e3-4157-90fd-c8b9177f0efc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:31.347177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51d3dc-0a2d-4fa2-9da8-2f1f5231d6d0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:31.359169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e73f96-b5a8-49fa-b171-5dcdc508c2bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:31.362889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e73f96-b5a8-49fa-b171-5dcdc508c2bc service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:31.363285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e73f96-b5a8-49fa-b171-5dcdc508c2bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:31.363677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e73f96-b5a8-49fa-b171-5dcdc508c2bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:31.454855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98df3c9-a4bc-400f-a271-d9186c161dc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:31.458022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98df3c9-a4bc-400f-a271-d9186c161dc7 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:31.458395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98df3c9-a4bc-400f-a271-d9186c161dc7 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:31.458753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98df3c9-a4bc-400f-a271-d9186c161dc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:31.475377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e73f96-b5a8-49fa-b171-5dcdc508c2bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:31.477261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e73f96-b5a8-49fa-b171-5dcdc508c2bc service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:31.492544 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1153/2314] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:31.507951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e73f96-b5a8-49fa-b171-5dcdc508c2bc service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:31.532114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e73f96-b5a8-49fa-b171-5dcdc508c2bc service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:31.545693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c5973-093d-4988-b133-cf8bdc8e8b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:31.546661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c5973-093d-4988-b133-cf8bdc8e8b08 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:31.546661 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c5973-093d-4988-b133-cf8bdc8e8b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:31.546929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c5973-093d-4988-b133-cf8bdc8e8b08 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:31.547608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c5973-093d-4988-b133-cf8bdc8e8b08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:31.554642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c5973-093d-4988-b133-cf8bdc8e8b08 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:31.590078 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98df3c9-a4bc-400f-a271-d9186c161dc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:31.591834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98df3c9-a4bc-400f-a271-d9186c161dc7 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:31.608335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1162/2315] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:24:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:24:31.650256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1154/2316] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:31 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:31.651588 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98df3c9-a4bc-400f-a271-d9186c161dc7 service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:31.673499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98df3c9-a4bc-400f-a271-d9186c161dc7 service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:31.684578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dd35f-a308-487c-ab97-c8c6dd09d767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:31.685613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dd35f-a308-487c-ab97-c8c6dd09d767 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:31.686057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dd35f-a308-487c-ab97-c8c6dd09d767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:31.687842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dd35f-a308-487c-ab97-c8c6dd09d767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:31.689397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537dd35f-a308-487c-ab97-c8c6dd09d767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:31.691455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537dd35f-a308-487c-ab97-c8c6dd09d767 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:31.700473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:31.706310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1163/2317] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:24:31.710210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c5973-093d-4988-b133-cf8bdc8e8b08 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:31.726670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c5973-093d-4988-b133-cf8bdc8e8b08 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:31.738876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5c500-dfd3-4943-8717-6ab6e15b06d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:31.739644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5c500-dfd3-4943-8717-6ab6e15b06d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:31.739970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5c500-dfd3-4943-8717-6ab6e15b06d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:31.740277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5c500-dfd3-4943-8717-6ab6e15b06d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:31.740948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5c500-dfd3-4943-8717-6ab6e15b06d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:31.747840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5c500-dfd3-4943-8717-6ab6e15b06d3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:31.774139 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1155/2318] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:24:31.779247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dd35f-a308-487c-ab97-c8c6dd09d767 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:31.791342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dd35f-a308-487c-ab97-c8c6dd09d767 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:31.806604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff494b-43ff-4e90-8504-01ed5e7c173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:31.807302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff494b-43ff-4e90-8504-01ed5e7c173f admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:31.807594 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff494b-43ff-4e90-8504-01ed5e7c173f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:31.807904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff494b-43ff-4e90-8504-01ed5e7c173f admin admin] PATH_INFO: `/v3/projects/a7704d42875d460ba8e93544dde70950/users/e75d6709d4d8425c98cf9a31a2947556/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:31.857439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff494b-43ff-4e90-8504-01ed5e7c173f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7704d42875d460ba8e93544dde70950, user_id=e75d6709d4d8425c98cf9a31a2947556, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:31.871941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff494b-43ff-4e90-8504-01ed5e7c173f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:31.988288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1164/2319] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:31 2022] PUT /identity/v3/projects/a7704d42875d460ba8e93544dde70950/users/e75d6709d4d8425c98cf9a31a2947556/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:31.999230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a68d6-59c6-4fa5-8f1c-214a3c9e88f3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:31.999574 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a68d6-59c6-4fa5-8f1c-214a3c9e88f3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:31.999937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a68d6-59c6-4fa5-8f1c-214a3c9e88f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:32.043070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9e8a68d6-59c6-4fa5-8f1c-214a3c9e88f3 None None] MFA Rules not processed for user `e75d6709d4d8425c98cf9a31a2947556`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:32.044915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9e8a68d6-59c6-4fa5-8f1c-214a3c9e88f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:32.176789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1156/2320] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:32.181152 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff494b-43ff-4e90-8504-01ed5e7c173f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:32.288729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff494b-43ff-4e90-8504-01ed5e7c173f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:32.304124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69067608-ddd1-4092-9d52-8162ffcb494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:32.304896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69067608-ddd1-4092-9d52-8162ffcb494f admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:32.305343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69067608-ddd1-4092-9d52-8162ffcb494f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:32.305715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69067608-ddd1-4092-9d52-8162ffcb494f admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:32.306839 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69067608-ddd1-4092-9d52-8162ffcb494f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:32.309653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69067608-ddd1-4092-9d52-8162ffcb494f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:32.429508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1165/2321] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:32 2022] POST /identity/v3/projects => generated 375 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:32.431362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a68d6-59c6-4fa5-8f1c-214a3c9e88f3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:32.458159 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a68d6-59c6-4fa5-8f1c-214a3c9e88f3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:32.581004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d675f-b54c-4439-bdfd-f45c389ab7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:32.582023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d675f-b54c-4439-bdfd-f45c389ab7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:32.582946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d675f-b54c-4439-bdfd-f45c389ab7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:32.583658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d675f-b54c-4439-bdfd-f45c389ab7ab admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:32.590443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d675f-b54c-4439-bdfd-f45c389ab7ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:32.590443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d675f-b54c-4439-bdfd-f45c389ab7ab admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:32.679664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1157/2322] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:32 2022] POST /identity/v3/users => generated 294 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:32.681028 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69067608-ddd1-4092-9d52-8162ffcb494f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:32.692580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69067608-ddd1-4092-9d52-8162ffcb494f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:32.697174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd410d1a-07d7-4410-a96b-49b74b9a003e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:32.697797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd410d1a-07d7-4410-a96b-49b74b9a003e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:32.698081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd410d1a-07d7-4410-a96b-49b74b9a003e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:32.698381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd410d1a-07d7-4410-a96b-49b74b9a003e admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:32.699084 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd410d1a-07d7-4410-a96b-49b74b9a003e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:32.700727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd410d1a-07d7-4410-a96b-49b74b9a003e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:32.713675 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1166/2323] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:32.717963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d675f-b54c-4439-bdfd-f45c389ab7ab admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:32.728929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d675f-b54c-4439-bdfd-f45c389ab7ab admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:32.735258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844db20-b003-4a0a-a1d1-01aea11a76a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:32.736046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844db20-b003-4a0a-a1d1-01aea11a76a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:32.736427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844db20-b003-4a0a-a1d1-01aea11a76a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:32.737033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844db20-b003-4a0a-a1d1-01aea11a76a2 admin admin] PATH_INFO: `/v3/projects/fa35711ae152415bab168a0e41cd08e5/users/af1eb94db06e44faa0b7595ca0e4b5d4/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:32.753229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:32.763557 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844db20-b003-4a0a-a1d1-01aea11a76a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa35711ae152415bab168a0e41cd08e5, user_id=af1eb94db06e44faa0b7595ca0e4b5d4, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:32.766902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844db20-b003-4a0a-a1d1-01aea11a76a2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:32.836151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f8673-5485-46b4-ac45-e63018ef6029 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:32.836526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f8673-5485-46b4-ac45-e63018ef6029 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:32.836879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f8673-5485-46b4-ac45-e63018ef6029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:32.862398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-807f8673-5485-46b4-ac45-e63018ef6029 None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:32.863888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-807f8673-5485-46b4-ac45-e63018ef6029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:32.909596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1158/2324] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:32 2022] PUT /identity/v3/projects/fa35711ae152415bab168a0e41cd08e5/users/af1eb94db06e44faa0b7595ca0e4b5d4/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:24:32.911093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844db20-b003-4a0a-a1d1-01aea11a76a2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:32.984522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:32.998713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1167/2325] 10.222.0.25 () {58 vars in 967 bytes} [Sat Aug 6 01:24:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:24:33.013961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f8673-5485-46b4-ac45-e63018ef6029 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:33.029973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844db20-b003-4a0a-a1d1-01aea11a76a2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:33.041400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4b1a-5060-47a2-b5b2-3c82377df0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:33.041964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4b1a-5060-47a2-b5b2-3c82377df0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:33.042251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4b1a-5060-47a2-b5b2-3c82377df0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:33.042536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4b1a-5060-47a2-b5b2-3c82377df0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:33.043271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4b1a-5060-47a2-b5b2-3c82377df0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:33.045004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4b1a-5060-47a2-b5b2-3c82377df0b8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:33.060357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1159/2326] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:33.061637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4b1a-5060-47a2-b5b2-3c82377df0b8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:33.073407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4b1a-5060-47a2-b5b2-3c82377df0b8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:33.079599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d426c6-d208-4576-a527-bfd1455b522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:33.081719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d426c6-d208-4576-a527-bfd1455b522a admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:33.082180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d426c6-d208-4576-a527-bfd1455b522a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:33.082613 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d426c6-d208-4576-a527-bfd1455b522a admin admin] PATH_INFO: `/v3/projects/fa35711ae152415bab168a0e41cd08e5/users/af1eb94db06e44faa0b7595ca0e4b5d4/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:33.086167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d426c6-d208-4576-a527-bfd1455b522a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa35711ae152415bab168a0e41cd08e5, user_id=af1eb94db06e44faa0b7595ca0e4b5d4, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:33.089244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d426c6-d208-4576-a527-bfd1455b522a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:33.118569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1160/2327] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:33 2022] PUT /identity/v3/projects/fa35711ae152415bab168a0e41cd08e5/users/af1eb94db06e44faa0b7595ca0e4b5d4/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:33.118912 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f8673-5485-46b4-ac45-e63018ef6029 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:33.119042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89347066-8d28-493b-862f-41d6a4ab4710 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:33.119349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89347066-8d28-493b-862f-41d6a4ab4710 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:33.119638 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89347066-8d28-493b-862f-41d6a4ab4710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:33.170910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-89347066-8d28-493b-862f-41d6a4ab4710 None None] MFA Rules not processed for user `af1eb94db06e44faa0b7595ca0e4b5d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:33.172263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-89347066-8d28-493b-862f-41d6a4ab4710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:33.215983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89179a-1ff8-4645-8e29-790c1fd13b65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:33.216705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89179a-1ff8-4645-8e29-790c1fd13b65 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:33.217374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89179a-1ff8-4645-8e29-790c1fd13b65 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:33.217806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89179a-1ff8-4645-8e29-790c1fd13b65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:33.260126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1161/2328] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:33.275800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89347066-8d28-493b-862f-41d6a4ab4710 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:33.278364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-89347066-8d28-493b-862f-41d6a4ab4710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:33.333805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:33.336979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89179a-1ff8-4645-8e29-790c1fd13b65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:33.342261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89179a-1ff8-4645-8e29-790c1fd13b65 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:33.354611 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1168/2329] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:24:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:24:33.388958 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89347066-8d28-493b-862f-41d6a4ab4710 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:33.394645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a40f1-c368-4ddb-bc8f-4e0d7e78de70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:33.395389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a40f1-c368-4ddb-bc8f-4e0d7e78de70 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:33.395931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a40f1-c368-4ddb-bc8f-4e0d7e78de70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:33.396275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a40f1-c368-4ddb-bc8f-4e0d7e78de70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:33.469988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a40f1-c368-4ddb-bc8f-4e0d7e78de70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:33.471492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a40f1-c368-4ddb-bc8f-4e0d7e78de70 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:33.482611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1162/2330] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:33.676357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:34.140693 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236fc6-c431-495d-8e12-99d11cda73fc None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:34.141179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236fc6-c431-495d-8e12-99d11cda73fc None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:34.141273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6236fc6-c431-495d-8e12-99d11cda73fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:34.180559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d6236fc6-c431-495d-8e12-99d11cda73fc None None] MFA Rules not processed for user `d327cdb7279d487289c113b25bf59426`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:34.182075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d6236fc6-c431-495d-8e12-99d11cda73fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:34.220657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053cda0-0936-4ba3-9c5e-af319e567999 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:34.221195 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053cda0-0936-4ba3-9c5e-af319e567999 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:34.221475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4053cda0-0936-4ba3-9c5e-af319e567999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:34.246378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-4053cda0-0936-4ba3-9c5e-af319e567999 None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:34.249804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4053cda0-0936-4ba3-9c5e-af319e567999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:34.259415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1163/2331] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:24:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:24:34.293025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053cda0-0936-4ba3-9c5e-af319e567999 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:34.316328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4053cda0-0936-4ba3-9c5e-af319e567999 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:34.323556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:34.329425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6af57-ab94-425e-9b82-bd5a4e6e5fe0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:34.330059 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6af57-ab94-425e-9b82-bd5a4e6e5fe0 service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:34.330360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6af57-ab94-425e-9b82-bd5a4e6e5fe0 service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:34.330805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6af57-ab94-425e-9b82-bd5a4e6e5fe0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:34.338576 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1169/2332] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:34.352057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6af57-ab94-425e-9b82-bd5a4e6e5fe0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:34.353084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6af57-ab94-425e-9b82-bd5a4e6e5fe0 service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:34.354786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6236fc6-c431-495d-8e12-99d11cda73fc None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:34.357295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d6236fc6-c431-495d-8e12-99d11cda73fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:34.363077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1164/2333] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:24:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 01:24:34.461441 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6236fc6-c431-495d-8e12-99d11cda73fc None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:34.467957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e955d7-8250-446f-8677-a61e3e8c5606 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:34.479806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e955d7-8250-446f-8677-a61e3e8c5606 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:34.479806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e955d7-8250-446f-8677-a61e3e8c5606 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:34.479806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e955d7-8250-446f-8677-a61e3e8c5606 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:34.479806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e5e955d7-8250-446f-8677-a61e3e8c5606 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:34.550032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e955d7-8250-446f-8677-a61e3e8c5606 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:34.554094 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e955d7-8250-446f-8677-a61e3e8c5606 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:34.565048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1170/2334] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:34.576913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:35.165430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d65a5f-7042-4029-a12d-91809016991c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:35.165729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d65a5f-7042-4029-a12d-91809016991c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:35.165997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d65a5f-7042-4029-a12d-91809016991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:35.221795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a9d65a5f-7042-4029-a12d-91809016991c None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:35.227307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a9d65a5f-7042-4029-a12d-91809016991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:35.248506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1165/2335] 10.222.0.25 () {58 vars in 967 bytes} [Sat Aug 6 01:24:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:35.262339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e955d7-8250-446f-8677-a61e3e8c5606 service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:35.281231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e955d7-8250-446f-8677-a61e3e8c5606 service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:35.301441 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316f819-2b54-4408-a10b-45d9b00a0c36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:35.302043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316f819-2b54-4408-a10b-45d9b00a0c36 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:35.302339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316f819-2b54-4408-a10b-45d9b00a0c36 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:35.302625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316f819-2b54-4408-a10b-45d9b00a0c36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:35.323100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316f819-2b54-4408-a10b-45d9b00a0c36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:35.326998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316f819-2b54-4408-a10b-45d9b00a0c36 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:35.342562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1171/2336] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:24:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:24:37.200586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d65a5f-7042-4029-a12d-91809016991c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:37.204501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a9d65a5f-7042-4029-a12d-91809016991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:37.226766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d65a5f-7042-4029-a12d-91809016991c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:37.239509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112316b-73cb-429a-b5a2-0e45ef58affe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:37.240453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112316b-73cb-429a-b5a2-0e45ef58affe service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:37.240858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112316b-73cb-429a-b5a2-0e45ef58affe service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:37.241243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112316b-73cb-429a-b5a2-0e45ef58affe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:37.349752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112316b-73cb-429a-b5a2-0e45ef58affe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:37.355177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112316b-73cb-429a-b5a2-0e45ef58affe service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:37.395964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1166/2337] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:38.514408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321cc7e-5b48-4e73-a08a-9ea76bc57090 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:38.516126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321cc7e-5b48-4e73-a08a-9ea76bc57090 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:38.516632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321cc7e-5b48-4e73-a08a-9ea76bc57090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:38.563181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4321cc7e-5b48-4e73-a08a-9ea76bc57090 None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:38.565423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4321cc7e-5b48-4e73-a08a-9ea76bc57090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:38.574829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1172/2338] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:24:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:38.589529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112316b-73cb-429a-b5a2-0e45ef58affe service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:38.608019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112316b-73cb-429a-b5a2-0e45ef58affe service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:38.614085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6430197-409d-4ef8-a1d3-13963bcb05e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:38.614924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6430197-409d-4ef8-a1d3-13963bcb05e1 service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:38.615489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6430197-409d-4ef8-a1d3-13963bcb05e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:38.615976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6430197-409d-4ef8-a1d3-13963bcb05e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:38.636972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6430197-409d-4ef8-a1d3-13963bcb05e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:38.638590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6430197-409d-4ef8-a1d3-13963bcb05e1 service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:38.665017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1167/2339] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:24:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:24:39.331430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef458-4193-4bf2-b37f-8a3c352de3a3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:39.331814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef458-4193-4bf2-b37f-8a3c352de3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:39.332112 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef458-4193-4bf2-b37f-8a3c352de3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:39.386210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bdaef458-4193-4bf2-b37f-8a3c352de3a3 None None] MFA Rules not processed for user `aa886e3c6aac40c8975bad310f65e517`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:39.389275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bdaef458-4193-4bf2-b37f-8a3c352de3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:39.403553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1173/2340] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:39.472623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6430197-409d-4ef8-a1d3-13963bcb05e1 service swift] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:39.474384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d6430197-409d-4ef8-a1d3-13963bcb05e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:39.489626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:39.665674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6430197-409d-4ef8-a1d3-13963bcb05e1 service swift] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:39.670601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:39.672020 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:39.673083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:39.674113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] PATH_INFO: `/v3/users/5aed2648ebea49559750faeb4c0006eb` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:39.675754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aed2648ebea49559750faeb4c0006eb)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:39.678271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:39.949896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aed2648ebea49559750faeb4c0006eb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:39.967483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aed2648ebea49559750faeb4c0006eb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:40.074689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aed2648ebea49559750faeb4c0006eb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:40.075479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aed2648ebea49559750faeb4c0006eb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:40.078546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aed2648ebea49559750faeb4c0006eb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:40.165172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1168/2341] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:39 2022] DELETE /identity/v3/users/5aed2648ebea49559750faeb4c0006eb => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:40.166962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7071ed-a8e9-49cb-a3ff-9604dbdf8ff9 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:40.167442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7071ed-a8e9-49cb-a3ff-9604dbdf8ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:40.167564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7071ed-a8e9-49cb-a3ff-9604dbdf8ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:40.209198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4f7071ed-a8e9-49cb-a3ff-9604dbdf8ff9 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:40.216105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4f7071ed-a8e9-49cb-a3ff-9604dbdf8ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:40.217020 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:40.218549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:40.275757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:40.291843 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1174/2342] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:40.309199 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7071ed-a8e9-49cb-a3ff-9604dbdf8ff9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:40.310894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4f7071ed-a8e9-49cb-a3ff-9604dbdf8ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:40.399210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1b0cc-a1c9-41a0-9281-670f96ab9551 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:40.413296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f166775-2421-422f-b692-bcb69a38ddc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:40.413946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f166775-2421-422f-b692-bcb69a38ddc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:40.414228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f166775-2421-422f-b692-bcb69a38ddc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:40.414503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f166775-2421-422f-b692-bcb69a38ddc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:40.418294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0f166775-2421-422f-b692-bcb69a38ddc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:40.464670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7071ed-a8e9-49cb-a3ff-9604dbdf8ff9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:40.474619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad213094-8d0c-42ae-aa57-5586bc35a83b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:40.474619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad213094-8d0c-42ae-aa57-5586bc35a83b service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:40.474619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad213094-8d0c-42ae-aa57-5586bc35a83b service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:40.474619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad213094-8d0c-42ae-aa57-5586bc35a83b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:40.506853 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad213094-8d0c-42ae-aa57-5586bc35a83b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:40.508351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad213094-8d0c-42ae-aa57-5586bc35a83b service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:40.523671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:40.530038 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1175/2343] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:40.598234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f166775-2421-422f-b692-bcb69a38ddc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:40.605041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f166775-2421-422f-b692-bcb69a38ddc6 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:40.633625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1169/2344] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:24:41.096810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad213094-8d0c-42ae-aa57-5586bc35a83b service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:41.098299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ad213094-8d0c-42ae-aa57-5586bc35a83b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:41.121786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad213094-8d0c-42ae-aa57-5586bc35a83b service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:41.132095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:41.133301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:41.134543 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:41.134543 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] PATH_INFO: `/v3/users/736713aabecd425087e1fa8bfebb368a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:41.139595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=736713aabecd425087e1fa8bfebb368a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:41.144166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:41.246645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f166775-2421-422f-b692-bcb69a38ddc6 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:41.250619 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0f166775-2421-422f-b692-bcb69a38ddc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:41.303489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f166775-2421-422f-b692-bcb69a38ddc6 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:41.308976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:41.309651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:41.309942 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:41.310219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] PATH_INFO: `/v3/projects/c70ade3d7f604830a6b48ba0386dce67` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:41.311899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c70ade3d7f604830a6b48ba0386dce67)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:41.314827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:41.389826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '736713aabecd425087e1fa8bfebb368a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:41.412417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '736713aabecd425087e1fa8bfebb368a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:41.469924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c70ade3d7f604830a6b48ba0386dce67', 'request_id': 'req-5020f80b-1881-46d6-8fcb-eabe18fe459b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:41.483514 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c70ade3d7f604830a6b48ba0386dce67', 'request_id': 'req-5020f80b-1881-46d6-8fcb-eabe18fe459b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:41.535113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '736713aabecd425087e1fa8bfebb368a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:41.541155 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '736713aabecd425087e1fa8bfebb368a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:41.576045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2615d080-45eb-4f37-8b67-07ac39e835f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '736713aabecd425087e1fa8bfebb368a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:41.588067 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1176/2345] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:41 2022] DELETE /identity/v3/users/736713aabecd425087e1fa8bfebb368a => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:41.594559 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17ac21-fd3f-44d0-8b31-25df02c3ab0c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:41.595131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17ac21-fd3f-44d0-8b31-25df02c3ab0c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:41.595557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17ac21-fd3f-44d0-8b31-25df02c3ab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:41.626779 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ef17ac21-fd3f-44d0-8b31-25df02c3ab0c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:41.628599 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ef17ac21-fd3f-44d0-8b31-25df02c3ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:41.640807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c70ade3d7f604830a6b48ba0386dce67', 'request_id': 'req-5020f80b-1881-46d6-8fcb-eabe18fe459b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:41.657890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] The token cache is being invalidate because project c70ade3d7f604830a6b48ba0386dce67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:41.658384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:41.661783 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-5020f80b-1881-46d6-8fcb-eabe18fe459b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:41.669513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1170/2346] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:41 2022] DELETE /identity/v3/projects/c70ade3d7f604830a6b48ba0386dce67 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:41.675330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a185c94-d4df-4a45-8841-7ce9f24d1c8a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:41.684134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a185c94-d4df-4a45-8841-7ce9f24d1c8a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:41.684134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a185c94-d4df-4a45-8841-7ce9f24d1c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:41.724432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8a185c94-d4df-4a45-8841-7ce9f24d1c8a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:41.725982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8a185c94-d4df-4a45-8841-7ce9f24d1c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:41.736036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1177/2347] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:41.745086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1171/2348] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:24:41.746797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17ac21-fd3f-44d0-8b31-25df02c3ab0c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:41.748451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ef17ac21-fd3f-44d0-8b31-25df02c3ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:41.752828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2ff7e-1c30-44ef-8b5d-d37b26726dd4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:41.753338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2ff7e-1c30-44ef-8b5d-d37b26726dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:41.753625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2ff7e-1c30-44ef-8b5d-d37b26726dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:41.767340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:41.816055 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-35b2ff7e-1c30-44ef-8b5d-d37b26726dd4 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:41.820085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-35b2ff7e-1c30-44ef-8b5d-d37b26726dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:41.836917 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17ac21-fd3f-44d0-8b31-25df02c3ab0c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:41.841032 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b5337-77bc-45f6-a807-96d804117622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:41.842043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5337-77bc-45f6-a807-96d804117622 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:41.842522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5337-77bc-45f6-a807-96d804117622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:41.843025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5337-77bc-45f6-a807-96d804117622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:41.857733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b5337-77bc-45f6-a807-96d804117622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:41.859620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b5337-77bc-45f6-a807-96d804117622 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:41.871678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1178/2349] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:41.887921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1172/2350] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:41.892096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b5337-77bc-45f6-a807-96d804117622 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:41.911145 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b5337-77bc-45f6-a807-96d804117622 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:41.934007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ebee6-41ef-47f6-994c-7021008450f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:41.940243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ebee6-41ef-47f6-994c-7021008450f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:41.943093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ebee6-41ef-47f6-994c-7021008450f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:41.943873 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ebee6-41ef-47f6-994c-7021008450f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:41.949016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ebee6-41ef-47f6-994c-7021008450f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:41.949016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ebee6-41ef-47f6-994c-7021008450f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:41.966339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ebee6-41ef-47f6-994c-7021008450f7 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:42.005972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1179/2351] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:24:42.011074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2ff7e-1c30-44ef-8b5d-d37b26726dd4 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:42.032654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2ff7e-1c30-44ef-8b5d-d37b26726dd4 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:42.050384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bea2e-d7a9-4cc3-afcc-4122ff513031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:42.052583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bea2e-d7a9-4cc3-afcc-4122ff513031 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:42.053193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bea2e-d7a9-4cc3-afcc-4122ff513031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:42.053701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bea2e-d7a9-4cc3-afcc-4122ff513031 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:42.054644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61bea2e-d7a9-4cc3-afcc-4122ff513031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:42.057761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61bea2e-d7a9-4cc3-afcc-4122ff513031 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:42.225260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1173/2352] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:42 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:42.226515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ebee6-41ef-47f6-994c-7021008450f7 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:42.246291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ebee6-41ef-47f6-994c-7021008450f7 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:42.444849 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596795b-d03b-4df4-97e3-e0bd0c2ce3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:42.445632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596795b-d03b-4df4-97e3-e0bd0c2ce3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:42.446072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596795b-d03b-4df4-97e3-e0bd0c2ce3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:42.446662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596795b-d03b-4df4-97e3-e0bd0c2ce3d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:42.450980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596795b-d03b-4df4-97e3-e0bd0c2ce3d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:42.456259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596795b-d03b-4df4-97e3-e0bd0c2ce3d2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:42.556037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1180/2353] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:42 2022] POST /identity/v3/users => generated 293 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:42.560058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bea2e-d7a9-4cc3-afcc-4122ff513031 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:42.591120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bea2e-d7a9-4cc3-afcc-4122ff513031 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:42.612526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbbae2-0bdf-4c35-837b-6a6d6034043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:42.613278 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbbae2-0bdf-4c35-837b-6a6d6034043f admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:42.616252 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbbae2-0bdf-4c35-837b-6a6d6034043f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:42.616797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbbae2-0bdf-4c35-837b-6a6d6034043f admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:42.617745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbbae2-0bdf-4c35-837b-6a6d6034043f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:42.626225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbbae2-0bdf-4c35-837b-6a6d6034043f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:42.651168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1174/2354] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:42.654673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596795b-d03b-4df4-97e3-e0bd0c2ce3d2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:42.677952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596795b-d03b-4df4-97e3-e0bd0c2ce3d2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:42.706990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3f540-970f-4989-8b1d-eff079b8ca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:42.711758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3f540-970f-4989-8b1d-eff079b8ca03 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:42.712783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3f540-970f-4989-8b1d-eff079b8ca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:42.716664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3f540-970f-4989-8b1d-eff079b8ca03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:42.716664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3f540-970f-4989-8b1d-eff079b8ca03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:42.738148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3f540-970f-4989-8b1d-eff079b8ca03 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:42.762299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1181/2355] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:42.767088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbbae2-0bdf-4c35-837b-6a6d6034043f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:42.790776 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbbae2-0bdf-4c35-837b-6a6d6034043f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:42.793776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3f540-970f-4989-8b1d-eff079b8ca03 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:42.796344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-39d3f540-970f-4989-8b1d-eff079b8ca03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:42.816137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be0975-2398-42c2-9f43-6201fb873cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:42.816137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be0975-2398-42c2-9f43-6201fb873cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:42.816137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be0975-2398-42c2-9f43-6201fb873cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:42.820693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be0975-2398-42c2-9f43-6201fb873cdc admin admin] PATH_INFO: `/v3/projects/019c42b12f444a35a1617fc7383c212c/users/ee469b9be6bf4ef192560551e817b385/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:42.833758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3f540-970f-4989-8b1d-eff079b8ca03 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:42.848427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:42.851027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:42.852590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:42.852972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] PATH_INFO: `/v3/projects/fc4141860179465eb3db2ad4b3c16eec` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:42.856504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4141860179465eb3db2ad4b3c16eec)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:42.883005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:42.884620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be0975-2398-42c2-9f43-6201fb873cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=019c42b12f444a35a1617fc7383c212c, user_id=ee469b9be6bf4ef192560551e817b385, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:42.893807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be0975-2398-42c2-9f43-6201fb873cdc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:43.011423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1175/2356] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:42 2022] PUT /identity/v3/projects/019c42b12f444a35a1617fc7383c212c/users/ee469b9be6bf4ef192560551e817b385/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:43.018099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698de98-6861-4f7d-9991-ff23549e757b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:43.025011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698de98-6861-4f7d-9991-ff23549e757b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:43.025622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698de98-6861-4f7d-9991-ff23549e757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:43.096863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d698de98-6861-4f7d-9991-ff23549e757b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:43.098982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d698de98-6861-4f7d-9991-ff23549e757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:43.119643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4141860179465eb3db2ad4b3c16eec', 'request_id': 'req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:43.141139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4141860179465eb3db2ad4b3c16eec', 'request_id': 'req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:43.163079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4141860179465eb3db2ad4b3c16eec', 'request_id': 'req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:43.222863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1176/2357] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:43.238037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698de98-6861-4f7d-9991-ff23549e757b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:43.310482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] The token cache is being invalidate because project fc4141860179465eb3db2ad4b3c16eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:43.310820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:43.313641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3f1e018b-3bc1-46e1-8d1b-c8890ef7cf38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:43.318070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:43.323481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1182/2358] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:42 2022] DELETE /identity/v3/projects/fc4141860179465eb3db2ad4b3c16eec => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:43.333179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ae69-5483-4b6e-87c0-25774c57896e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:43.333565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ae69-5483-4b6e-87c0-25774c57896e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:43.333941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7ae69-5483-4b6e-87c0-25774c57896e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:43.398398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698de98-6861-4f7d-9991-ff23549e757b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:43.407883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cf162-fd19-4853-a749-73f1cc902f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:43.408535 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cf162-fd19-4853-a749-73f1cc902f0e service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:43.409235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-e7f7ae69-5483-4b6e-87c0-25774c57896e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:43.410184 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cf162-fd19-4853-a749-73f1cc902f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:43.410465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cf162-fd19-4853-a749-73f1cc902f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:43.411804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e7f7ae69-5483-4b6e-87c0-25774c57896e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:43.414903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fb2cf162-fd19-4853-a749-73f1cc902f0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:43.455602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1183/2359] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:43.473405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09eadd8-4ced-4fa8-9acc-27549cd1536f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:43.473715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09eadd8-4ced-4fa8-9acc-27549cd1536f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:43.474206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09eadd8-4ced-4fa8-9acc-27549cd1536f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:43.486520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2cf162-fd19-4853-a749-73f1cc902f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:43.489866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2cf162-fd19-4853-a749-73f1cc902f0e service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:43.521716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1177/2360] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:43.522527 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d09eadd8-4ced-4fa8-9acc-27549cd1536f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:43.523902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d09eadd8-4ced-4fa8-9acc-27549cd1536f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:43.548034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1184/2361] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:43.550763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cf162-fd19-4853-a749-73f1cc902f0e service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:43.569496 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cf162-fd19-4853-a749-73f1cc902f0e service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:43.586318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd70689-32a4-4a5c-a517-7192e8458921 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:43.586690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd70689-32a4-4a5c-a517-7192e8458921 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:43.586928 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd70689-32a4-4a5c-a517-7192e8458921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:43.597458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fee7d3-3299-4fa2-99f1-5001cedd91fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:43.605589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fee7d3-3299-4fa2-99f1-5001cedd91fa admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:43.605589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fee7d3-3299-4fa2-99f1-5001cedd91fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:43.605589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fee7d3-3299-4fa2-99f1-5001cedd91fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:43.605589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fee7d3-3299-4fa2-99f1-5001cedd91fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:43.605589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fee7d3-3299-4fa2-99f1-5001cedd91fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:43.611306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fee7d3-3299-4fa2-99f1-5001cedd91fa admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:43.658555 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1178/2362] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:24:43.662477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fee7d3-3299-4fa2-99f1-5001cedd91fa admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:43.673627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4fd70689-32a4-4a5c-a517-7192e8458921 None None] MFA Rules not processed for user `42047fe93eb24adbbc24844a75ed21f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:43.675017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4fd70689-32a4-4a5c-a517-7192e8458921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:43.690503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fee7d3-3299-4fa2-99f1-5001cedd91fa admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:43.693172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c1246-a3a9-4a26-b306-42489dcca5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:43.694225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c1246-a3a9-4a26-b306-42489dcca5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:43.694748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c1246-a3a9-4a26-b306-42489dcca5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:43.695241 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c1246-a3a9-4a26-b306-42489dcca5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:43.696172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c1246-a3a9-4a26-b306-42489dcca5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:43.699132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c1246-a3a9-4a26-b306-42489dcca5b6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:43.838710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1179/2363] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:43 2022] POST /identity/v3/projects => generated 391 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:43.843819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c1246-a3a9-4a26-b306-42489dcca5b6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:43.861643 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1185/2364] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:24:43.870432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd70689-32a4-4a5c-a517-7192e8458921 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:43.870432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4fd70689-32a4-4a5c-a517-7192e8458921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:43.886102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd70689-32a4-4a5c-a517-7192e8458921 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:43.978825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c1246-a3a9-4a26-b306-42489dcca5b6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:43.990565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36a15e-ff60-4d32-b0e0-d7ce0a8bf353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:43.992205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36a15e-ff60-4d32-b0e0-d7ce0a8bf353 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:43.992617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36a15e-ff60-4d32-b0e0-d7ce0a8bf353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:43.992986 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36a15e-ff60-4d32-b0e0-d7ce0a8bf353 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:43.993915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36a15e-ff60-4d32-b0e0-d7ce0a8bf353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:43.996659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36a15e-ff60-4d32-b0e0-d7ce0a8bf353 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:44.001200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587034e7-0e09-4b6b-9993-6939bc29852d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:44.001907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587034e7-0e09-4b6b-9993-6939bc29852d admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:44.002194 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587034e7-0e09-4b6b-9993-6939bc29852d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:44.002474 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587034e7-0e09-4b6b-9993-6939bc29852d admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:44.003223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587034e7-0e09-4b6b-9993-6939bc29852d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:44.005230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587034e7-0e09-4b6b-9993-6939bc29852d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:44.070140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1180/2365] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:43 2022] POST /identity/v3/users => generated 296 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:44.074149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36a15e-ff60-4d32-b0e0-d7ce0a8bf353 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:44.086852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36a15e-ff60-4d32-b0e0-d7ce0a8bf353 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:44.104140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1847009-0cf3-4422-b63e-13fbe53a2d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:44.104880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1847009-0cf3-4422-b63e-13fbe53a2d63 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:44.105280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1847009-0cf3-4422-b63e-13fbe53a2d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:44.105685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1847009-0cf3-4422-b63e-13fbe53a2d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:44.106530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1847009-0cf3-4422-b63e-13fbe53a2d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:44.108578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1847009-0cf3-4422-b63e-13fbe53a2d63 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:44.127531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1186/2366] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:43 2022] POST /identity/v3/projects => generated 379 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:44.131735 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587034e7-0e09-4b6b-9993-6939bc29852d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:44.142924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1181/2367] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:24:44.144028 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1847009-0cf3-4422-b63e-13fbe53a2d63 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:44.170961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1847009-0cf3-4422-b63e-13fbe53a2d63 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:44.276171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df3161-7367-492f-ba1d-1ac13bca3cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:44.276171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3161-7367-492f-ba1d-1ac13bca3cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:44.276171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3161-7367-492f-ba1d-1ac13bca3cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:44.276171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3161-7367-492f-ba1d-1ac13bca3cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:44.277119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df3161-7367-492f-ba1d-1ac13bca3cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:44.278895 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df3161-7367-492f-ba1d-1ac13bca3cdf admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:44.301513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1182/2368] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:44.307109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df3161-7367-492f-ba1d-1ac13bca3cdf admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:44.311835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587034e7-0e09-4b6b-9993-6939bc29852d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:44.326542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b4e12-e6de-4fcc-8d1d-046e579ca82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:44.327387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b4e12-e6de-4fcc-8d1d-046e579ca82f admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:44.333501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b4e12-e6de-4fcc-8d1d-046e579ca82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:44.333501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b4e12-e6de-4fcc-8d1d-046e579ca82f admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:44.333501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b4e12-e6de-4fcc-8d1d-046e579ca82f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:44.333501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b4e12-e6de-4fcc-8d1d-046e579ca82f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:44.335423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df3161-7367-492f-ba1d-1ac13bca3cdf admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:44.345316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ef2a8-c8b0-4515-b5c2-182412664b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:44.346221 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ef2a8-c8b0-4515-b5c2-182412664b85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:44.346473 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ef2a8-c8b0-4515-b5c2-182412664b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:44.346755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ef2a8-c8b0-4515-b5c2-182412664b85 admin admin] PATH_INFO: `/v3/projects/db0a2b8d78be41feb3105985d4cbd00f/users/002e6ee7a6ae42faaba1d3623e3279ef/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:44.369385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ef2a8-c8b0-4515-b5c2-182412664b85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db0a2b8d78be41feb3105985d4cbd00f, user_id=002e6ee7a6ae42faaba1d3623e3279ef, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:44.372182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ef2a8-c8b0-4515-b5c2-182412664b85 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:44.406918 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:44.435900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1187/2369] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:44 2022] POST /identity/v3/users => generated 296 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:44.445915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b4e12-e6de-4fcc-8d1d-046e579ca82f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:44.462611 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b4e12-e6de-4fcc-8d1d-046e579ca82f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:44.472938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1183/2370] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:44 2022] PUT /identity/v3/projects/db0a2b8d78be41feb3105985d4cbd00f/users/002e6ee7a6ae42faaba1d3623e3279ef/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:44.484452 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f1c7c-1891-4d14-839f-9f9393a80ae1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:44.484977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f1c7c-1891-4d14-839f-9f9393a80ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:44.485666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f1c7c-1891-4d14-839f-9f9393a80ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:44.620825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-344f1c7c-1891-4d14-839f-9f9393a80ae1 None None] MFA Rules not processed for user `002e6ee7a6ae42faaba1d3623e3279ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:44.637894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-344f1c7c-1891-4d14-839f-9f9393a80ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:44.704244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4757a-c6bb-4c3a-886a-87c7f99f6863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:44.705085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4757a-c6bb-4c3a-886a-87c7f99f6863 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:44.705470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4757a-c6bb-4c3a-886a-87c7f99f6863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:44.705846 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4757a-c6bb-4c3a-886a-87c7f99f6863 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:44.707130 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4757a-c6bb-4c3a-886a-87c7f99f6863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:44.709692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4757a-c6bb-4c3a-886a-87c7f99f6863 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:44.737824 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1188/2371] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:24:44.759006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4757a-c6bb-4c3a-886a-87c7f99f6863 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:44.792850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4757a-c6bb-4c3a-886a-87c7f99f6863 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:44.809315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce14ca-ce66-40f6-83af-83648ed8c63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:44.813965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce14ca-ce66-40f6-83af-83648ed8c63c admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:44.813965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce14ca-ce66-40f6-83af-83648ed8c63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:44.813965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce14ca-ce66-40f6-83af-83648ed8c63c admin admin] PATH_INFO: `/v3/projects/c7cf8e9bf49647a2884bddbea672f616/users/c67f5a5e953c42e8a33f5b92a006e479/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:44.871468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce14ca-ce66-40f6-83af-83648ed8c63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7cf8e9bf49647a2884bddbea672f616, user_id=c67f5a5e953c42e8a33f5b92a006e479, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:44.874308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce14ca-ce66-40f6-83af-83648ed8c63c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:44.891644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1184/2372] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:44.892931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f1c7c-1891-4d14-839f-9f9393a80ae1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:44.894705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-344f1c7c-1891-4d14-839f-9f9393a80ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:45.040980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1189/2373] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:44 2022] PUT /identity/v3/projects/c7cf8e9bf49647a2884bddbea672f616/users/c67f5a5e953c42e8a33f5b92a006e479/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:45.044003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f1c7c-1891-4d14-839f-9f9393a80ae1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:45.045973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce14ca-ce66-40f6-83af-83648ed8c63c admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:45.138067 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc831f08-bb7c-4a94-b0b4-c7b1e607c0be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:45.141103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc831f08-bb7c-4a94-b0b4-c7b1e607c0be service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:45.141395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc831f08-bb7c-4a94-b0b4-c7b1e607c0be service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:45.141678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc831f08-bb7c-4a94-b0b4-c7b1e607c0be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:45.224803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce14ca-ce66-40f6-83af-83648ed8c63c admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:45.235172 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f874160-31e6-4141-bbb0-95f9525af8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:45.235847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f874160-31e6-4141-bbb0-95f9525af8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:45.236136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f874160-31e6-4141-bbb0-95f9525af8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:45.236415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f874160-31e6-4141-bbb0-95f9525af8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:45.237127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f874160-31e6-4141-bbb0-95f9525af8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:45.238953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f874160-31e6-4141-bbb0-95f9525af8e9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:45.277822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1190/2374] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:45.283385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f874160-31e6-4141-bbb0-95f9525af8e9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:45.298436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f874160-31e6-4141-bbb0-95f9525af8e9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:45.320607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb171cb-86af-4395-ade4-0b849d432e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:45.321339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb171cb-86af-4395-ade4-0b849d432e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:45.322495 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb171cb-86af-4395-ade4-0b849d432e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:45.322792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb171cb-86af-4395-ade4-0b849d432e52 admin admin] PATH_INFO: `/v3/projects/c7cf8e9bf49647a2884bddbea672f616/users/c67f5a5e953c42e8a33f5b92a006e479/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:45.330627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb171cb-86af-4395-ade4-0b849d432e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7cf8e9bf49647a2884bddbea672f616, user_id=c67f5a5e953c42e8a33f5b92a006e479, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:45.335884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb171cb-86af-4395-ade4-0b849d432e52 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:45.340644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc831f08-bb7c-4a94-b0b4-c7b1e607c0be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:45.344720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc831f08-bb7c-4a94-b0b4-c7b1e607c0be service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:45.373744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1191/2375] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:45 2022] PUT /identity/v3/projects/c7cf8e9bf49647a2884bddbea672f616/users/c67f5a5e953c42e8a33f5b92a006e479/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:45.377849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1185/2376] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:45.590679 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975ce50-959d-426c-8050-16ad9a009875 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:45.590979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975ce50-959d-426c-8050-16ad9a009875 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:45.591432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975ce50-959d-426c-8050-16ad9a009875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:45.664455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7975ce50-959d-426c-8050-16ad9a009875 None None] MFA Rules not processed for user `1c96cc61c2b84b3b81ed6c6738f9773e`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:45.668829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7975ce50-959d-426c-8050-16ad9a009875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:45.854818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1186/2377] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:24:46.270893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:46.509606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb171cb-86af-4395-ade4-0b849d432e52 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:46.512929 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9cb171cb-86af-4395-ade4-0b849d432e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:46.733263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb171cb-86af-4395-ade4-0b849d432e52 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:46.764724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:46.765791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:46.766286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:46.766867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] PATH_INFO: `/v3/users/9a6dbaea48f547e99b40c7acd114489e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:46.769205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6dbaea48f547e99b40c7acd114489e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:46.772496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:47.121762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6dbaea48f547e99b40c7acd114489e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:47.153325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6dbaea48f547e99b40c7acd114489e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:47.159921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:47.249312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6dbaea48f547e99b40c7acd114489e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:47.249312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6dbaea48f547e99b40c7acd114489e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:47.270684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6dbaea48f547e99b40c7acd114489e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:47.286720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1192/2378] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:46 2022] DELETE /identity/v3/users/9a6dbaea48f547e99b40c7acd114489e => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:47.292797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53780f4c-3bcf-4b92-abc9-339010856d05 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:47.293369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53780f4c-3bcf-4b92-abc9-339010856d05 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:47.293573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53780f4c-3bcf-4b92-abc9-339010856d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:47.347122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-53780f4c-3bcf-4b92-abc9-339010856d05 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:47.348714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-53780f4c-3bcf-4b92-abc9-339010856d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:47.517818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1187/2379] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:47.525830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:47.527505 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:47.630459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:47.651581 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fc1ea-dc9f-4ade-b7c7-ea54c5f8e195 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:47.657482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc842c-c455-471c-b772-5c8d6b58a3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:47.659320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc842c-c455-471c-b772-5c8d6b58a3bc service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:47.661242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc842c-c455-471c-b772-5c8d6b58a3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:47.665128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc842c-c455-471c-b772-5c8d6b58a3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:47.674354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc842c-c455-471c-b772-5c8d6b58a3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:47.676509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc842c-c455-471c-b772-5c8d6b58a3bc service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:47.689837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1193/2380] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:48.320784 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:48.557335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53780f4c-3bcf-4b92-abc9-339010856d05 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:48.559679 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-53780f4c-3bcf-4b92-abc9-339010856d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:48.589190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53780f4c-3bcf-4b92-abc9-339010856d05 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:48.604107 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:48.613973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:48.613973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:48.616786 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] PATH_INFO: `/v3/projects/999498337156490dbbe38504a38c8035` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:48.620815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=999498337156490dbbe38504a38c8035)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:48.642596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:48.970566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999498337156490dbbe38504a38c8035', 'request_id': 'req-2687ed3a-211b-4c8b-841f-320558ea6340'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:48.995265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999498337156490dbbe38504a38c8035', 'request_id': 'req-2687ed3a-211b-4c8b-841f-320558ea6340'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:49.046153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:49.119579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999498337156490dbbe38504a38c8035', 'request_id': 'req-2687ed3a-211b-4c8b-841f-320558ea6340'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:49.145070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] The token cache is being invalidate because project 999498337156490dbbe38504a38c8035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:49.145764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:49.146764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:49.150369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1188/2381] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:48 2022] DELETE /identity/v3/projects/999498337156490dbbe38504a38c8035 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:49.154645 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc842c-c455-471c-b772-5c8d6b58a3bc service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:49.168553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc842c-c455-471c-b772-5c8d6b58a3bc service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:49.323752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:49.324619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:49.325001 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:49.325435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] PATH_INFO: `/v3/users/e75d6709d4d8425c98cf9a31a2947556` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:49.328597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e75d6709d4d8425c98cf9a31a2947556)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:49.331135 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:49.639390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e75d6709d4d8425c98cf9a31a2947556'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:49.659502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e75d6709d4d8425c98cf9a31a2947556'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:49.733514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e75d6709d4d8425c98cf9a31a2947556'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:49.735276 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e75d6709d4d8425c98cf9a31a2947556'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:49.753970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e75d6709d4d8425c98cf9a31a2947556'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:49.767430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1194/2382] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:49 2022] DELETE /identity/v3/users/e75d6709d4d8425c98cf9a31a2947556 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:24:49.818165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:51.004671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:51.006094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:51.030570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2687ed3a-211b-4c8b-841f-320558ea6340 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:51.252209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:51.252880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:51.253339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:51.253776 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] PATH_INFO: `/v3/projects/a7704d42875d460ba8e93544dde70950` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:51.255659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7704d42875d460ba8e93544dde70950)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:51.257663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:51.515841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7704d42875d460ba8e93544dde70950', 'request_id': 'req-6a59604d-6023-4ae1-807c-ca03c2bdc95a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:51.538728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7704d42875d460ba8e93544dde70950', 'request_id': 'req-6a59604d-6023-4ae1-807c-ca03c2bdc95a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:51.626977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7704d42875d460ba8e93544dde70950', 'request_id': 'req-6a59604d-6023-4ae1-807c-ca03c2bdc95a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:51.648770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] The token cache is being invalidate because project a7704d42875d460ba8e93544dde70950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:51.649562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:51.650124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:51.654209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1189/2383] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:51 2022] DELETE /identity/v3/projects/a7704d42875d460ba8e93544dde70950 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:51.658291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:51.659855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:51.690261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d6eb9-394a-4d4f-838b-c3118bdee9fa admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:51.823854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:51.829433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:51.829433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:51.829433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] PATH_INFO: `/v3/users/af1eb94db06e44faa0b7595ca0e4b5d4` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:51.829433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1eb94db06e44faa0b7595ca0e4b5d4)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:51.830189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:52.047066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1eb94db06e44faa0b7595ca0e4b5d4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:52.073896 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1eb94db06e44faa0b7595ca0e4b5d4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:52.155526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1eb94db06e44faa0b7595ca0e4b5d4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:52.156946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1eb94db06e44faa0b7595ca0e4b5d4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:52.188983 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b8709b8e-5974-4445-a6a7-65a80708ac96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1eb94db06e44faa0b7595ca0e4b5d4'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:52.201869 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1195/2384] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:51 2022] DELETE /identity/v3/users/af1eb94db06e44faa0b7595ca0e4b5d4 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:54.236937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:54.238520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:54.258809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59604d-6023-4ae1-807c-ca03c2bdc95a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:54.412580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:54.413412 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:54.413801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:54.414396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] PATH_INFO: `/v3/projects/fa35711ae152415bab168a0e41cd08e5` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:54.417985 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa35711ae152415bab168a0e41cd08e5)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:54.420103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:54.429789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:54.617322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa35711ae152415bab168a0e41cd08e5', 'request_id': 'req-847eab9f-98eb-4cb4-bff7-7e365c014b46'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:54.623510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa35711ae152415bab168a0e41cd08e5', 'request_id': 'req-847eab9f-98eb-4cb4-bff7-7e365c014b46'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:54.698514 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa35711ae152415bab168a0e41cd08e5', 'request_id': 'req-847eab9f-98eb-4cb4-bff7-7e365c014b46'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:54.722738 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] The token cache is being invalidate because project fa35711ae152415bab168a0e41cd08e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:54.723169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:54.728907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-847eab9f-98eb-4cb4-bff7-7e365c014b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:54.730741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1190/2385] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:54 2022] DELETE /identity/v3/projects/fa35711ae152415bab168a0e41cd08e5 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:54.738075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d23e27-020e-4a19-92fe-c78d329d723c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:54.739242 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d23e27-020e-4a19-92fe-c78d329d723c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:54.739731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d23e27-020e-4a19-92fe-c78d329d723c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:54.756419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:54.782164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-90d23e27-020e-4a19-92fe-c78d329d723c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:54.786308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-90d23e27-020e-4a19-92fe-c78d329d723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:54.983514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1196/2386] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:54.991944 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86066c-ea96-49cb-9d1f-89bfdc5aad8b None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:54.992526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86066c-ea96-49cb-9d1f-89bfdc5aad8b None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:54.992919 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86066c-ea96-49cb-9d1f-89bfdc5aad8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:55.055185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ad86066c-ea96-49cb-9d1f-89bfdc5aad8b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:55.057135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ad86066c-ea96-49cb-9d1f-89bfdc5aad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:55.080588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1191/2387] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:55.085115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d23e27-020e-4a19-92fe-c78d329d723c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:55.099229 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d23e27-020e-4a19-92fe-c78d329d723c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:55.139091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef680ba-720c-4474-bfdc-c3c0bc65a550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:55.139944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef680ba-720c-4474-bfdc-c3c0bc65a550 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:55.140371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef680ba-720c-4474-bfdc-c3c0bc65a550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:55.140750 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef680ba-720c-4474-bfdc-c3c0bc65a550 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:55.141585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef680ba-720c-4474-bfdc-c3c0bc65a550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:24:55.141997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef680ba-720c-4474-bfdc-c3c0bc65a550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:55.143819 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef680ba-720c-4474-bfdc-c3c0bc65a550 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:55.189978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1197/2388] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:24:55.195939 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86066c-ea96-49cb-9d1f-89bfdc5aad8b None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:55.206881 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86066c-ea96-49cb-9d1f-89bfdc5aad8b None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:55.221751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f28a7a-8163-4018-8576-e0f4ef3c3c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:55.222580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f28a7a-8163-4018-8576-e0f4ef3c3c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:55.222973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f28a7a-8163-4018-8576-e0f4ef3c3c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:55.223363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f28a7a-8163-4018-8576-e0f4ef3c3c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:55.224312 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f28a7a-8163-4018-8576-e0f4ef3c3c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:55.230018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f28a7a-8163-4018-8576-e0f4ef3c3c9d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:55.368302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1192/2389] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:55 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:55.371632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef680ba-720c-4474-bfdc-c3c0bc65a550 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:55.394804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef680ba-720c-4474-bfdc-c3c0bc65a550 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:55.561753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3da41-6e92-411e-b8d4-f54b69269aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:55.563683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3da41-6e92-411e-b8d4-f54b69269aca admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:55.564489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3da41-6e92-411e-b8d4-f54b69269aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:55.566000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3da41-6e92-411e-b8d4-f54b69269aca admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:55.567061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3da41-6e92-411e-b8d4-f54b69269aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:55.580829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3da41-6e92-411e-b8d4-f54b69269aca admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:55.745692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1198/2390] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:55 2022] POST /identity/v3/users => generated 286 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:55.749739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f28a7a-8163-4018-8576-e0f4ef3c3c9d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:55.771193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f28a7a-8163-4018-8576-e0f4ef3c3c9d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:55.784362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1d15f-db85-424c-8173-fd59e274cffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:55.785382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1d15f-db85-424c-8173-fd59e274cffc admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:55.797847 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1d15f-db85-424c-8173-fd59e274cffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:55.801819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1d15f-db85-424c-8173-fd59e274cffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:55.804513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1d15f-db85-424c-8173-fd59e274cffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:55.810696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1d15f-db85-424c-8173-fd59e274cffc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:55.842129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1193/2391] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:55.845500 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3da41-6e92-411e-b8d4-f54b69269aca admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:55.863087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3da41-6e92-411e-b8d4-f54b69269aca admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:55.881634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7b2ab-3c56-49ce-8b8c-2e759b6479e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:55.884526 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7b2ab-3c56-49ce-8b8c-2e759b6479e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:55.887304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7b2ab-3c56-49ce-8b8c-2e759b6479e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:55.888408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7b2ab-3c56-49ce-8b8c-2e759b6479e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:55.903142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7b2ab-3c56-49ce-8b8c-2e759b6479e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:55.908835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7b2ab-3c56-49ce-8b8c-2e759b6479e8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:55.944281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1199/2392] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:55.949283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1d15f-db85-424c-8173-fd59e274cffc admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:55.980501 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1d15f-db85-424c-8173-fd59e274cffc admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:55.993995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73f74b-8c1a-4d81-b973-72604aee5b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:55.995684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73f74b-8c1a-4d81-b973-72604aee5b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:55.996215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73f74b-8c1a-4d81-b973-72604aee5b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:55.997317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73f74b-8c1a-4d81-b973-72604aee5b55 admin admin] PATH_INFO: `/v3/projects/580b0344924448a4890aff7054cb0207/users/4652399968094c5b9e5f1dc3e17eaebf/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:56.018780 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73f74b-8c1a-4d81-b973-72604aee5b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=580b0344924448a4890aff7054cb0207, user_id=4652399968094c5b9e5f1dc3e17eaebf, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:56.023056 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73f74b-8c1a-4d81-b973-72604aee5b55 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:56.106477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1194/2393] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:55 2022] PUT /identity/v3/projects/580b0344924448a4890aff7054cb0207/users/4652399968094c5b9e5f1dc3e17eaebf/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:56.209297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc4a3d-9d8d-485f-aebc-aac1c35e96d4 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:56.209816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc4a3d-9d8d-485f-aebc-aac1c35e96d4 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:56.210207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc4a3d-9d8d-485f-aebc-aac1c35e96d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:56.295419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-28fc4a3d-9d8d-485f-aebc-aac1c35e96d4 None None] MFA Rules not processed for user `4652399968094c5b9e5f1dc3e17eaebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:56.305403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-28fc4a3d-9d8d-485f-aebc-aac1c35e96d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:56.514243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:56.535006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1200/2394] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:56.538988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73f74b-8c1a-4d81-b973-72604aee5b55 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:56.540901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4a73f74b-8c1a-4d81-b973-72604aee5b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:56.676712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73f74b-8c1a-4d81-b973-72604aee5b55 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:56.681045 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63771c77-84dc-4e05-a1b1-df279add6cf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:56.681926 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63771c77-84dc-4e05-a1b1-df279add6cf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:56.682320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63771c77-84dc-4e05-a1b1-df279add6cf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:56.682705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63771c77-84dc-4e05-a1b1-df279add6cf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:56.694670 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63771c77-84dc-4e05-a1b1-df279add6cf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:56.696759 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63771c77-84dc-4e05-a1b1-df279add6cf4 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:56.729222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1195/2395] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:24:56.754259 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc4a3d-9d8d-485f-aebc-aac1c35e96d4 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:56.845797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc4a3d-9d8d-485f-aebc-aac1c35e96d4 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:56.858189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:56.858189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:56.858189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:56.858189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] PATH_INFO: `/v3/users/4652399968094c5b9e5f1dc3e17eaebf` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:56.860333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4652399968094c5b9e5f1dc3e17eaebf)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:56.862640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:57.084839 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4652399968094c5b9e5f1dc3e17eaebf'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:57.123897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4652399968094c5b9e5f1dc3e17eaebf'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:57.213175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4652399968094c5b9e5f1dc3e17eaebf'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:57.213960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4652399968094c5b9e5f1dc3e17eaebf'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:57.290130 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bd729b6b-1dab-4d26-946e-4533fc5cdfaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4652399968094c5b9e5f1dc3e17eaebf'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:57.316544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1201/2396] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:24:56 2022] DELETE /identity/v3/users/4652399968094c5b9e5f1dc3e17eaebf => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:57.317654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde936d2-b3a4-4668-9755-084a7026cb14 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:57.317995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde936d2-b3a4-4668-9755-084a7026cb14 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:57.318275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde936d2-b3a4-4668-9755-084a7026cb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:57.377081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-dde936d2-b3a4-4668-9755-084a7026cb14 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:57.382704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-dde936d2-b3a4-4668-9755-084a7026cb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:57.479377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:57.545741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53107f-7113-4414-959a-780218e2601f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:57.546019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53107f-7113-4414-959a-780218e2601f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:57.546338 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53107f-7113-4414-959a-780218e2601f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:57.567741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1196/2397] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:24:57.600089 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde936d2-b3a4-4668-9755-084a7026cb14 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:57.601974 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-dde936d2-b3a4-4668-9755-084a7026cb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:57.669663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6e53107f-7113-4414-959a-780218e2601f None None] MFA Rules not processed for user `ee469b9be6bf4ef192560551e817b385`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:57.675680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6e53107f-7113-4414-959a-780218e2601f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:57.859359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1202/2398] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:24:57.863766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53107f-7113-4414-959a-780218e2601f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:57.866402 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6e53107f-7113-4414-959a-780218e2601f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:57.884660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde936d2-b3a4-4668-9755-084a7026cb14 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:57.890766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0f358-94fa-4d69-94ec-575a24e57025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:57.891797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0f358-94fa-4d69-94ec-575a24e57025 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:57.892285 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0f358-94fa-4d69-94ec-575a24e57025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:57.892457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0f358-94fa-4d69-94ec-575a24e57025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:57.910984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0f358-94fa-4d69-94ec-575a24e57025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:57.911239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0f358-94fa-4d69-94ec-575a24e57025 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:57.918318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53107f-7113-4414-959a-780218e2601f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:57.939323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1197/2399] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:24:57.948938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337728e-12fe-42b7-b962-87e4077abb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:57.953910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337728e-12fe-42b7-b962-87e4077abb4f admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:57.954200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337728e-12fe-42b7-b962-87e4077abb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:57.957460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337728e-12fe-42b7-b962-87e4077abb4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:57.960747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337728e-12fe-42b7-b962-87e4077abb4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:57.965071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337728e-12fe-42b7-b962-87e4077abb4f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:57.989725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:58.155405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1203/2400] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:24:57 2022] POST /identity/v3/projects => generated 385 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:24:58.160099 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0f358-94fa-4d69-94ec-575a24e57025 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:58.180318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7af9e-4642-4461-80c4-abee922ab9d5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:58.180430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7af9e-4642-4461-80c4-abee922ab9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:58.180704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7af9e-4642-4461-80c4-abee922ab9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:58.191117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0f358-94fa-4d69-94ec-575a24e57025 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:58.236728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-73b7af9e-4642-4461-80c4-abee922ab9d5 None None] MFA Rules not processed for user `c67f5a5e953c42e8a33f5b92a006e479`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:24:58.239587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-73b7af9e-4642-4461-80c4-abee922ab9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:58.354113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584ac9c-c70c-48ca-a8bb-2faa6ae1d69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:58.358040 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584ac9c-c70c-48ca-a8bb-2faa6ae1d69c admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:58.358395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584ac9c-c70c-48ca-a8bb-2faa6ae1d69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:58.358686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584ac9c-c70c-48ca-a8bb-2faa6ae1d69c admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:58.361269 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584ac9c-c70c-48ca-a8bb-2faa6ae1d69c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:58.364052 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584ac9c-c70c-48ca-a8bb-2faa6ae1d69c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:58.403937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1204/2401] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:24:58 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:24:58.427576 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7af9e-4642-4461-80c4-abee922ab9d5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:58.435369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-73b7af9e-4642-4461-80c4-abee922ab9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:58.516801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1198/2402] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:24:58 2022] POST /identity/v3/users => generated 299 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:24:58.521357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584ac9c-c70c-48ca-a8bb-2faa6ae1d69c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:58.584715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584ac9c-c70c-48ca-a8bb-2faa6ae1d69c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:58.594546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf34145-b7b3-4d51-9e07-6cef4764f3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:58.595537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf34145-b7b3-4d51-9e07-6cef4764f3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:58.596391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf34145-b7b3-4d51-9e07-6cef4764f3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:58.597714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf34145-b7b3-4d51-9e07-6cef4764f3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:58.600565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf34145-b7b3-4d51-9e07-6cef4764f3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:58.602335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf34145-b7b3-4d51-9e07-6cef4764f3fb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:58.639995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1199/2403] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:58.645140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf34145-b7b3-4d51-9e07-6cef4764f3fb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:58.660303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7af9e-4642-4461-80c4-abee922ab9d5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:58.665843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf34145-b7b3-4d51-9e07-6cef4764f3fb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:58.673403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812768b3-5318-4a2e-a44a-5172d1086255 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:58.675733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812768b3-5318-4a2e-a44a-5172d1086255 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:58.676226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812768b3-5318-4a2e-a44a-5172d1086255 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:58.677169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812768b3-5318-4a2e-a44a-5172d1086255 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:58.681136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f2fe0-8762-4f5b-b296-696c1aa862ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:58.681481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-812768b3-5318-4a2e-a44a-5172d1086255 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:58.682323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f2fe0-8762-4f5b-b296-696c1aa862ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:58.683983 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f2fe0-8762-4f5b-b296-696c1aa862ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:58.684788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f2fe0-8762-4f5b-b296-696c1aa862ee admin admin] PATH_INFO: `/v3/projects/bbe0d42b9127436ab41248d9b6e174a9/users/79c5895434dd45469057b70a24c34a35/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:58.733175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f2fe0-8762-4f5b-b296-696c1aa862ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe0d42b9127436ab41248d9b6e174a9, user_id=79c5895434dd45469057b70a24c34a35, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:58.733175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f2fe0-8762-4f5b-b296-696c1aa862ee admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:58.801680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812768b3-5318-4a2e-a44a-5172d1086255 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:58.803293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812768b3-5318-4a2e-a44a-5172d1086255 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:58.813982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1205/2404] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:24:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:24:58.871181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1200/2405] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:58 2022] PUT /identity/v3/projects/bbe0d42b9127436ab41248d9b6e174a9/users/79c5895434dd45469057b70a24c34a35/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:58.872230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812768b3-5318-4a2e-a44a-5172d1086255 service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:58.886362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812768b3-5318-4a2e-a44a-5172d1086255 service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:58.897422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:58.949876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f2fe0-8762-4f5b-b296-696c1aa862ee admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:58.951746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-417f2fe0-8762-4f5b-b296-696c1aa862ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:24:58.994783 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8670279-6f67-4c93-9c86-980da7d40e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:58.995453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8670279-6f67-4c93-9c86-980da7d40e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:58.995738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8670279-6f67-4c93-9c86-980da7d40e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:58.996046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8670279-6f67-4c93-9c86-980da7d40e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:58.996778 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8670279-6f67-4c93-9c86-980da7d40e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:58.999139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8670279-6f67-4c93-9c86-980da7d40e5b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:59.041059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1206/2406] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:24:59.045733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8670279-6f67-4c93-9c86-980da7d40e5b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:24:59.070911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8670279-6f67-4c93-9c86-980da7d40e5b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:59.073606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:24:59.083430 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7617db1-235b-4069-9b08-002ed36444df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:59.084165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7617db1-235b-4069-9b08-002ed36444df admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:59.084456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7617db1-235b-4069-9b08-002ed36444df admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:59.084736 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7617db1-235b-4069-9b08-002ed36444df admin admin] PATH_INFO: `/v3/projects/bbe0d42b9127436ab41248d9b6e174a9/users/79c5895434dd45469057b70a24c34a35/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:59.092219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7617db1-235b-4069-9b08-002ed36444df admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe0d42b9127436ab41248d9b6e174a9, user_id=79c5895434dd45469057b70a24c34a35, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:59.099041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7617db1-235b-4069-9b08-002ed36444df admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:59.131105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1207/2407] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:24:59 2022] PUT /identity/v3/projects/bbe0d42b9127436ab41248d9b6e174a9/users/79c5895434dd45469057b70a24c34a35/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:24:59.138142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f2fe0-8762-4f5b-b296-696c1aa862ee admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:24:59.309127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:24:59.310025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:24:59.310477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:24:59.310911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] PATH_INFO: `/v3/projects/580b0344924448a4890aff7054cb0207` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:24:59.312429 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=580b0344924448a4890aff7054cb0207)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:24:59.317391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:24:59.608120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '580b0344924448a4890aff7054cb0207', 'request_id': 'req-898a064c-62c1-4c9b-a41d-208a925a42a3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:59.615939 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '580b0344924448a4890aff7054cb0207', 'request_id': 'req-898a064c-62c1-4c9b-a41d-208a925a42a3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:59.733993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '580b0344924448a4890aff7054cb0207', 'request_id': 'req-898a064c-62c1-4c9b-a41d-208a925a42a3'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:59.767167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] The token cache is being invalidate because project 580b0344924448a4890aff7054cb0207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:24:59.767472 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:59.772488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:24:59.779995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1201/2408] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:24:58 2022] DELETE /identity/v3/projects/580b0344924448a4890aff7054cb0207 => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:25:09.332704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53885829-3c45-4f18-abfb-1d7be36fc1f7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:09.336838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53885829-3c45-4f18-abfb-1d7be36fc1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:09.337302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53885829-3c45-4f18-abfb-1d7be36fc1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:09.385903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-53885829-3c45-4f18-abfb-1d7be36fc1f7 None None] MFA Rules not processed for user `79c5895434dd45469057b70a24c34a35`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:09.387381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-53885829-3c45-4f18-abfb-1d7be36fc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:09.560824 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1208/2409] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:25:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:09.578766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:09.581337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:09.642321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53885829-3c45-4f18-abfb-1d7be36fc1f7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:09.648166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-53885829-3c45-4f18-abfb-1d7be36fc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:09.712117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a064c-62c1-4c9b-a41d-208a925a42a3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:09.721116 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf426e8-87fb-408a-8431-b1673852ba20 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a234a96eccd4f38b0213d7d0461ad24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:09.722029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf426e8-87fb-408a-8431-b1673852ba20 service glance] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:09.722550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf426e8-87fb-408a-8431-b1673852ba20 service glance] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:09.723813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf426e8-87fb-408a-8431-b1673852ba20 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:09.728259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cdf426e8-87fb-408a-8431-b1673852ba20 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:09.784880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53885829-3c45-4f18-abfb-1d7be36fc1f7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:09.802889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:09.802889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:09.802889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:09.802889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] PATH_INFO: `/v3/users/002e6ee7a6ae42faaba1d3623e3279ef` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:09.803747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002e6ee7a6ae42faaba1d3623e3279ef)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:09.803747 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:09.822396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:09.834972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:09.858641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf426e8-87fb-408a-8431-b1673852ba20 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:09.860834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf426e8-87fb-408a-8431-b1673852ba20 service glance] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:09.870104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1202/2410] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:10.094608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002e6ee7a6ae42faaba1d3623e3279ef'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:25:10.120681 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002e6ee7a6ae42faaba1d3623e3279ef'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:25:10.207271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002e6ee7a6ae42faaba1d3623e3279ef'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:25:10.208025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002e6ee7a6ae42faaba1d3623e3279ef'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:25:10.222449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-faa56af3-750c-48fe-a5b0-54b0193950c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002e6ee7a6ae42faaba1d3623e3279ef'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:25:10.230656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1209/2411] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:25:09 2022] DELETE /identity/v3/users/002e6ee7a6ae42faaba1d3623e3279ef => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:25:10.232460 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e7de8-907e-4d14-a4de-f97660ab4ff4 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:10.232755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e7de8-907e-4d14-a4de-f97660ab4ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:10.233304 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e7de8-907e-4d14-a4de-f97660ab4ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:10.235126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f0256-a660-4032-9d9c-5f285a8a3443 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:10.235605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f0256-a660-4032-9d9c-5f285a8a3443 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:10.235909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f0256-a660-4032-9d9c-5f285a8a3443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:10.255773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-019e7de8-907e-4d14-a4de-f97660ab4ff4 None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:10.256785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-4e8f0256-a660-4032-9d9c-5f285a8a3443 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:10.257261 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-019e7de8-907e-4d14-a4de-f97660ab4ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:10.258086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4e8f0256-a660-4032-9d9c-5f285a8a3443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:10.320578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1203/2412] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:25:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:10.334614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e7de8-907e-4d14-a4de-f97660ab4ff4 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:10.335971 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1210/2413] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:10.339573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-019e7de8-907e-4d14-a4de-f97660ab4ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:10.348625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f0256-a660-4032-9d9c-5f285a8a3443 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:10.354827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4e8f0256-a660-4032-9d9c-5f285a8a3443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:10.416277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f0256-a660-4032-9d9c-5f285a8a3443 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:10.420162 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2871a-2905-4310-b108-c734bcfaae9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:10.421022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2871a-2905-4310-b108-c734bcfaae9c service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:10.421335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2871a-2905-4310-b108-c734bcfaae9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:10.421614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2871a-2905-4310-b108-c734bcfaae9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:10.432909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2871a-2905-4310-b108-c734bcfaae9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:10.434674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2871a-2905-4310-b108-c734bcfaae9c service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:10.439026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e7de8-907e-4d14-a4de-f97660ab4ff4 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:10.451392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f160b-024b-46c8-a584-8628507e0e53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:10.453612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f160b-024b-46c8-a584-8628507e0e53 service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:10.454090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f160b-024b-46c8-a584-8628507e0e53 service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:10.454581 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1211/2414] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:25:10.455046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f160b-024b-46c8-a584-8628507e0e53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:10.466931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f160b-024b-46c8-a584-8628507e0e53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:10.469933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f160b-024b-46c8-a584-8628507e0e53 service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:10.484381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1204/2415] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:25:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 01:25:11.272829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2871a-2905-4310-b108-c734bcfaae9c service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:11.274451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-32f2871a-2905-4310-b108-c734bcfaae9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:11.296517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2871a-2905-4310-b108-c734bcfaae9c service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:11.308646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:11.309494 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:11.309959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:11.310500 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] PATH_INFO: `/v3/projects/db0a2b8d78be41feb3105985d4cbd00f` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:11.313257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db0a2b8d78be41feb3105985d4cbd00f)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:11.318400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:11.434952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:11.460218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db0a2b8d78be41feb3105985d4cbd00f', 'request_id': 'req-9f5f0f28-de8e-4a03-be3b-f8c89a499389'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:25:11.470355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db0a2b8d78be41feb3105985d4cbd00f', 'request_id': 'req-9f5f0f28-de8e-4a03-be3b-f8c89a499389'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:25:11.479649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db0a2b8d78be41feb3105985d4cbd00f', 'request_id': 'req-9f5f0f28-de8e-4a03-be3b-f8c89a499389'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:25:11.572210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] The token cache is being invalidate because project db0a2b8d78be41feb3105985d4cbd00f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:25:11.573024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:25:11.589131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9f5f0f28-de8e-4a03-be3b-f8c89a499389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:25:11.595096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1212/2416] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:25:11 2022] DELETE /identity/v3/projects/db0a2b8d78be41feb3105985d4cbd00f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:25:13.029081 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6db1a-0fe4-45aa-91b5-76b458a05e13 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:13.029507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6db1a-0fe4-45aa-91b5-76b458a05e13 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:13.029601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6db1a-0fe4-45aa-91b5-76b458a05e13 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:13.032295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1205/2417] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:13.046708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f9310-f0d3-4128-bcd6-e93112e967fd None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:13.046708 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f9310-f0d3-4128-bcd6-e93112e967fd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:13.047262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f9310-f0d3-4128-bcd6-e93112e967fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:13.141980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-155f9310-f0d3-4128-bcd6-e93112e967fd None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:13.145147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-155f9310-f0d3-4128-bcd6-e93112e967fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:13.349053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1213/2418] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:25:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:16.918242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6db1a-0fe4-45aa-91b5-76b458a05e13 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:16.934461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-02f6db1a-0fe4-45aa-91b5-76b458a05e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:17.015669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6db1a-0fe4-45aa-91b5-76b458a05e13 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:17.027341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacdc71-f6c3-4a31-b5a0-7fec530a6244 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a234a96eccd4f38b0213d7d0461ad24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:17.028328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacdc71-f6c3-4a31-b5a0-7fec530a6244 service glance] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:17.028615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacdc71-f6c3-4a31-b5a0-7fec530a6244 service glance] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:17.028892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacdc71-f6c3-4a31-b5a0-7fec530a6244 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:17.051357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacdc71-f6c3-4a31-b5a0-7fec530a6244 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:17.052964 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacdc71-f6c3-4a31-b5a0-7fec530a6244 service glance] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:17.060800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:17.066991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1206/2419] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:17.322035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0c872-e388-44f3-93fc-0f3a46049253 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:17.322035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0c872-e388-44f3-93fc-0f3a46049253 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:17.322656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0c872-e388-44f3-93fc-0f3a46049253 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:17.324108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1214/2420] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:17.329342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62fdf-8b1d-43e1-8224-826639da50b8 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:17.330192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62fdf-8b1d-43e1-8224-826639da50b8 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:17.330577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62fdf-8b1d-43e1-8224-826639da50b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:17.355582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a3c62fdf-8b1d-43e1-8224-826639da50b8 None None] MFA Rules not processed for user `2c14fb133a0f419390fb0d00c600bdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:17.357244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a3c62fdf-8b1d-43e1-8224-826639da50b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:17.427372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1207/2421] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:25:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:17.433336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0c872-e388-44f3-93fc-0f3a46049253 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:17.435510 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1aa0c872-e388-44f3-93fc-0f3a46049253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:17.460185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0c872-e388-44f3-93fc-0f3a46049253 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:17.472600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187aaea-ca42-4680-a696-b31f0633f9f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:17.473323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187aaea-ca42-4680-a696-b31f0633f9f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:17.474268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187aaea-ca42-4680-a696-b31f0633f9f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:17.474618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187aaea-ca42-4680-a696-b31f0633f9f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:17.487440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187aaea-ca42-4680-a696-b31f0633f9f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:17.489320 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187aaea-ca42-4680-a696-b31f0633f9f9 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:17.499897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1215/2422] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:17.618086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5076-571a-41d4-958a-55520c2a3171 None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:17.618650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5076-571a-41d4-958a-55520c2a3171 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:17.619072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc5076-571a-41d4-958a-55520c2a3171 None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:17.620673 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1208/2423] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:25:17.630591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22599220-20fe-495e-84a8-20ebd16a0980 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:17.630917 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22599220-20fe-495e-84a8-20ebd16a0980 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:17.631699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22599220-20fe-495e-84a8-20ebd16a0980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:17.676008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-22599220-20fe-495e-84a8-20ebd16a0980 None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:17.677252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-22599220-20fe-495e-84a8-20ebd16a0980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:17.704221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1216/2424] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:25:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:17.714515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc5076-571a-41d4-958a-55520c2a3171 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:17.716654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fafc5076-571a-41d4-958a-55520c2a3171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:17.744711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc5076-571a-41d4-958a-55520c2a3171 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:17.750875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd675977-638b-4ddd-9080-9115cc51c1bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:17.750875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd675977-638b-4ddd-9080-9115cc51c1bf service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:17.750875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd675977-638b-4ddd-9080-9115cc51c1bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:17.750875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd675977-638b-4ddd-9080-9115cc51c1bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:17.773038 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd675977-638b-4ddd-9080-9115cc51c1bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:17.774523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd675977-638b-4ddd-9080-9115cc51c1bf service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:17.790970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1209/2425] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:22.261704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65efaea-58f9-4ce9-8619-df46640540c1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:22.262301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65efaea-58f9-4ce9-8619-df46640540c1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:22.262649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65efaea-58f9-4ce9-8619-df46640540c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:22.286061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b65efaea-58f9-4ce9-8619-df46640540c1 None None] MFA Rules not processed for user `42047fe93eb24adbbc24844a75ed21f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:22.287375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b65efaea-58f9-4ce9-8619-df46640540c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:22.409380 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1217/2426] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:25:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:23.751474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b7c4c-3997-427f-b283-9bdd6ad1663e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:23.752051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b7c4c-3997-427f-b283-9bdd6ad1663e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:23.752165 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b7c4c-3997-427f-b283-9bdd6ad1663e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:23.799577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3b2b7c4c-3997-427f-b283-9bdd6ad1663e None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:23.800900 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3b2b7c4c-3997-427f-b283-9bdd6ad1663e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:23.880954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1210/2427] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:25:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:23.895500 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65efaea-58f9-4ce9-8619-df46640540c1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:23.897070 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b65efaea-58f9-4ce9-8619-df46640540c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:23.987022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:24.047158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65efaea-58f9-4ce9-8619-df46640540c1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:24.047158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdf4b2-be20-4258-8b66-b4f8deedc57d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:24.047158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdf4b2-be20-4258-8b66-b4f8deedc57d service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:24.047158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdf4b2-be20-4258-8b66-b4f8deedc57d service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:24.047627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdf4b2-be20-4258-8b66-b4f8deedc57d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:24.047627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdf4b2-be20-4258-8b66-b4f8deedc57d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:24.047627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdf4b2-be20-4258-8b66-b4f8deedc57d service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:24.054976 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1218/2428] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:25:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:25:24.977029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dc61d-ce5d-43ca-98a6-ffc4cfc7e7da None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:24.977518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dc61d-ce5d-43ca-98a6-ffc4cfc7e7da None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:24.977634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dc61d-ce5d-43ca-98a6-ffc4cfc7e7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:25.002101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7d0dc61d-ce5d-43ca-98a6-ffc4cfc7e7da None None] MFA Rules not processed for user `ee469b9be6bf4ef192560551e817b385`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:25.003453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7d0dc61d-ce5d-43ca-98a6-ffc4cfc7e7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:25.071269 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1211/2429] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:25:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:34.383436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=14 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:35.059110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=79 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:37.217742 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdf4b2-be20-4258-8b66-b4f8deedc57d service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:37.237801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdf4b2-be20-4258-8b66-b4f8deedc57d service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:37.243885 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2826-8df3-47e7-aca5-59bb6ba1b00c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:37.244914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2826-8df3-47e7-aca5-59bb6ba1b00c service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:37.245399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2826-8df3-47e7-aca5-59bb6ba1b00c service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:37.245992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a2826-8df3-47e7-aca5-59bb6ba1b00c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:37.248739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-631a2826-8df3-47e7-aca5-59bb6ba1b00c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:37.321797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a2826-8df3-47e7-aca5-59bb6ba1b00c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:37.323281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a2826-8df3-47e7-aca5-59bb6ba1b00c service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:37.333113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1219/2430] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:25:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:25:38.066996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56d03-39aa-4571-9398-3ec9bdffe906 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:38.067468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56d03-39aa-4571-9398-3ec9bdffe906 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:38.067753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56d03-39aa-4571-9398-3ec9bdffe906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:38.088696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-10f56d03-39aa-4571-9398-3ec9bdffe906 None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:38.090142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-10f56d03-39aa-4571-9398-3ec9bdffe906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:38.104029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1212/2431] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:25:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:38.125315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2826-8df3-47e7-aca5-59bb6ba1b00c service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:38.137673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a2826-8df3-47e7-aca5-59bb6ba1b00c service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:38.144026 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da072048-885f-46ad-9904-6ae0969f554d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:38.144659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da072048-885f-46ad-9904-6ae0969f554d service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:38.145043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da072048-885f-46ad-9904-6ae0969f554d service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:38.145423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da072048-885f-46ad-9904-6ae0969f554d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:38.154762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da072048-885f-46ad-9904-6ae0969f554d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:38.156748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da072048-885f-46ad-9904-6ae0969f554d service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:38.165858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1220/2432] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:25:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:25:48.083517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:48.162207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=36 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:25:50.192461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a48dc-8df7-4cdd-92cb-c45d245fce6d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:50.193289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a48dc-8df7-4cdd-92cb-c45d245fce6d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:50.193609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a48dc-8df7-4cdd-92cb-c45d245fce6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:50.214414 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5a0a48dc-8df7-4cdd-92cb-c45d245fce6d None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:25:50.215680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5a0a48dc-8df7-4cdd-92cb-c45d245fce6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:25:50.229330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1213/2433] 10.222.0.25 () {58 vars in 967 bytes} [Sat Aug 6 01:25:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:25:50.261174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da072048-885f-46ad-9904-6ae0969f554d service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:25:50.289297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da072048-885f-46ad-9904-6ae0969f554d service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:25:50.302829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb900ff-3c36-45c0-8e39-f4c1d598785f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:25:50.304419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb900ff-3c36-45c0-8e39-f4c1d598785f service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:25:50.304844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb900ff-3c36-45c0-8e39-f4c1d598785f service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:25:50.305129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb900ff-3c36-45c0-8e39-f4c1d598785f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:25:50.324870 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb900ff-3c36-45c0-8e39-f4c1d598785f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:25:50.326904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb900ff-3c36-45c0-8e39-f4c1d598785f service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:25:50.342275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1221/2434] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:25:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:26:00.208816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:00.337777 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=6 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:07.807867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a48dc-8df7-4cdd-92cb-c45d245fce6d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:07.817720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a48dc-8df7-4cdd-92cb-c45d245fce6d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:07.821316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ce56a-6d6b-4de2-ba63-70c5fed8f2ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:07.821933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ce56a-6d6b-4de2-ba63-70c5fed8f2ff service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:07.822220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ce56a-6d6b-4de2-ba63-70c5fed8f2ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:07.822506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ce56a-6d6b-4de2-ba63-70c5fed8f2ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:07.824286 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-278ce56a-6d6b-4de2-ba63-70c5fed8f2ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:07.886493 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ce56a-6d6b-4de2-ba63-70c5fed8f2ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:07.887977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ce56a-6d6b-4de2-ba63-70c5fed8f2ff service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:07.898113 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1214/2435] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:07 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:14.856088 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb900ff-3c36-45c0-8e39-f4c1d598785f service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:14.864465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb900ff-3c36-45c0-8e39-f4c1d598785f service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:14.868140 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285b1b6-a6fe-4da9-9e32-86f9dd9f67df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:14.868743 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285b1b6-a6fe-4da9-9e32-86f9dd9f67df service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:14.869024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285b1b6-a6fe-4da9-9e32-86f9dd9f67df service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:14.869316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285b1b6-a6fe-4da9-9e32-86f9dd9f67df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:14.871043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4285b1b6-a6fe-4da9-9e32-86f9dd9f67df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:14.936108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285b1b6-a6fe-4da9-9e32-86f9dd9f67df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:14.938947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285b1b6-a6fe-4da9-9e32-86f9dd9f67df service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:14.948609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1222/2436] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:17.893849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=30 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:24.944316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=30 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:36.614123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ce56a-6d6b-4de2-ba63-70c5fed8f2ff service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:26:36.615564 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-278ce56a-6d6b-4de2-ba63-70c5fed8f2ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:36.810668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ce56a-6d6b-4de2-ba63-70c5fed8f2ff service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:26:36.816498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e081b-9ade-40f8-a0e4-c90ae3e85392 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:26:36.817126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e081b-9ade-40f8-a0e4-c90ae3e85392 service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:36.817410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e081b-9ade-40f8-a0e4-c90ae3e85392 service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:36.817717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e081b-9ade-40f8-a0e4-c90ae3e85392 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:36.819684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f54e081b-9ade-40f8-a0e4-c90ae3e85392 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:36.832456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e081b-9ade-40f8-a0e4-c90ae3e85392 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:26:36.834100 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e081b-9ade-40f8-a0e4-c90ae3e85392 service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:26:36.844763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1215/2437] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:26:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:26:44.025200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a4f81-d981-4be2-9321-0a3df6a38a12 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:44.025671 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a4f81-d981-4be2-9321-0a3df6a38a12 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:44.025786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a4f81-d981-4be2-9321-0a3df6a38a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:44.045355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-829a4f81-d981-4be2-9321-0a3df6a38a12 None None] MFA Rules not processed for user `1c96cc61c2b84b3b81ed6c6738f9773e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:44.046557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-829a4f81-d981-4be2-9321-0a3df6a38a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:44.102823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1223/2438] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:26:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:26:46.840042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=32 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:54.096096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=25 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:26:58.897031 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fe122-7399-45b5-9222-2e38d2973fed None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:26:58.897506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fe122-7399-45b5-9222-2e38d2973fed None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:26:58.897703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fe122-7399-45b5-9222-2e38d2973fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:26:58.915885 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-059fe122-7399-45b5-9222-2e38d2973fed None None] MFA Rules not processed for user `2d417104bf4040d4ae9ccb08097b5255`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:26:58.917077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-059fe122-7399-45b5-9222-2e38d2973fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:26:58.969794 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1216/2439] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:26:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:08.963293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=25 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:28.822908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a4f81-d981-4be2-9321-0a3df6a38a12 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:28.824403 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-829a4f81-d981-4be2-9321-0a3df6a38a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:28.880464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a4f81-d981-4be2-9321-0a3df6a38a12 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:28.884023 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6050f70-5ab7-4220-ba8d-8558f0baedfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:28.884638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6050f70-5ab7-4220-ba8d-8558f0baedfc service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:28.884920 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6050f70-5ab7-4220-ba8d-8558f0baedfc service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:28.885196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6050f70-5ab7-4220-ba8d-8558f0baedfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:28.886932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a6050f70-5ab7-4220-ba8d-8558f0baedfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:28.899991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6050f70-5ab7-4220-ba8d-8558f0baedfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:28.901423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6050f70-5ab7-4220-ba8d-8558f0baedfc service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:28.912169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1224/2440] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:30.233668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fe122-7399-45b5-9222-2e38d2973fed None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:30.235148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-059fe122-7399-45b5-9222-2e38d2973fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:30.251439 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fe122-7399-45b5-9222-2e38d2973fed None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:30.257034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c87f1-8079-468c-a5e4-fd73e8280540 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:30.257643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c87f1-8079-468c-a5e4-fd73e8280540 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:30.257920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c87f1-8079-468c-a5e4-fd73e8280540 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:30.258190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c87f1-8079-468c-a5e4-fd73e8280540 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:30.259908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-554c87f1-8079-468c-a5e4-fd73e8280540 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:30.269914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c87f1-8079-468c-a5e4-fd73e8280540 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:30.271477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c87f1-8079-468c-a5e4-fd73e8280540 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:30.280356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1217/2441] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:31.801137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6050f70-5ab7-4220-ba8d-8558f0baedfc service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:31.810595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6050f70-5ab7-4220-ba8d-8558f0baedfc service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:31.814920 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a14df1-b3f6-4ad3-8f7d-ac10532828c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:31.815534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a14df1-b3f6-4ad3-8f7d-ac10532828c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:31.815842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a14df1-b3f6-4ad3-8f7d-ac10532828c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:31.816119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a14df1-b3f6-4ad3-8f7d-ac10532828c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:31.817913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c6a14df1-b3f6-4ad3-8f7d-ac10532828c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:31.831397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a14df1-b3f6-4ad3-8f7d-ac10532828c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:31.832845 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a14df1-b3f6-4ad3-8f7d-ac10532828c9 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:31.843633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1225/2442] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:40.276174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=6 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:41.838930 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=32 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:27:47.618927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c87f1-8079-468c-a5e4-fd73e8280540 service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:47.620931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-554c87f1-8079-468c-a5e4-fd73e8280540 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:47.646595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c87f1-8079-468c-a5e4-fd73e8280540 service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:47.654569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76b49a-4792-433d-b037-5cb0c0f7b125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:47.655512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76b49a-4792-433d-b037-5cb0c0f7b125 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:47.655976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76b49a-4792-433d-b037-5cb0c0f7b125 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:47.656411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76b49a-4792-433d-b037-5cb0c0f7b125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:47.658296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9f76b49a-4792-433d-b037-5cb0c0f7b125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:47.671869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76b49a-4792-433d-b037-5cb0c0f7b125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:47.673790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76b49a-4792-433d-b037-5cb0c0f7b125 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:47.686626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1218/2443] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:27:48.128888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a14df1-b3f6-4ad3-8f7d-ac10532828c9 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:48.140593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a14df1-b3f6-4ad3-8f7d-ac10532828c9 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:48.147435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2c701-77fe-4888-9219-ec600c9bda01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:48.148228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2c701-77fe-4888-9219-ec600c9bda01 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:48.148608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2c701-77fe-4888-9219-ec600c9bda01 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:48.148980 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2c701-77fe-4888-9219-ec600c9bda01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:48.150887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4a2c701-77fe-4888-9219-ec600c9bda01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:48.167139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2c701-77fe-4888-9219-ec600c9bda01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:48.168219 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2c701-77fe-4888-9219-ec600c9bda01 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:48.179413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1226/2444] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:51.116471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76b49a-4792-433d-b037-5cb0c0f7b125 service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:51.129959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76b49a-4792-433d-b037-5cb0c0f7b125 service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:51.136338 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe12ccd-913e-4c07-ab14-b2715d0052bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:51.136906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe12ccd-913e-4c07-ab14-b2715d0052bb service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:51.137183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe12ccd-913e-4c07-ab14-b2715d0052bb service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:51.137455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe12ccd-913e-4c07-ab14-b2715d0052bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:51.140014 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-efe12ccd-913e-4c07-ab14-b2715d0052bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:51.153614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe12ccd-913e-4c07-ab14-b2715d0052bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:51.153614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe12ccd-913e-4c07-ab14-b2715d0052bb service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:51.164061 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1219/2445] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:53.382111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7db8cf-09fa-4384-86da-54803867c5a6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:53.382111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7db8cf-09fa-4384-86da-54803867c5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:53.382111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7db8cf-09fa-4384-86da-54803867c5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:53.401670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5f7db8cf-09fa-4384-86da-54803867c5a6 None None] MFA Rules not processed for user `b07c458f1a5c4391aaaa79af49a236c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:27:53.402791 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5f7db8cf-09fa-4384-86da-54803867c5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:53.458612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1227/2446] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:27:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:27:53.987152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe12ccd-913e-4c07-ab14-b2715d0052bb service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:53.996824 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe12ccd-913e-4c07-ab14-b2715d0052bb service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:54.000361 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1183e1-b280-4886-bdeb-fa86a2b34a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:54.001025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1183e1-b280-4886-bdeb-fa86a2b34a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:54.001322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1183e1-b280-4886-bdeb-fa86a2b34a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:54.001597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1183e1-b280-4886-bdeb-fa86a2b34a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:54.003396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fa1183e1-b280-4886-bdeb-fa86a2b34a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:54.013135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1183e1-b280-4886-bdeb-fa86a2b34a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:54.014589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1183e1-b280-4886-bdeb-fa86a2b34a35 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:54.023220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1220/2447] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:54.565222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7db8cf-09fa-4384-86da-54803867c5a6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:54.584142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7db8cf-09fa-4384-86da-54803867c5a6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:54.594954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f96454-7cfe-45b3-8c11-cc15539602a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:54.595597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96454-7cfe-45b3-8c11-cc15539602a6 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:54.595911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96454-7cfe-45b3-8c11-cc15539602a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:54.596193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96454-7cfe-45b3-8c11-cc15539602a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:54.598051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-54f96454-7cfe-45b3-8c11-cc15539602a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:54.611131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f96454-7cfe-45b3-8c11-cc15539602a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:54.613164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f96454-7cfe-45b3-8c11-cc15539602a6 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:54.624197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1228/2448] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:56.127528 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1183e1-b280-4886-bdeb-fa86a2b34a35 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:56.138398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1183e1-b280-4886-bdeb-fa86a2b34a35 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:56.141989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934cb96-cb56-46dc-be7e-35c0196ec66d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:56.142756 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934cb96-cb56-46dc-be7e-35c0196ec66d service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:56.150714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934cb96-cb56-46dc-be7e-35c0196ec66d service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:56.150714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934cb96-cb56-46dc-be7e-35c0196ec66d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:56.150714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c934cb96-cb56-46dc-be7e-35c0196ec66d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:56.238068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934cb96-cb56-46dc-be7e-35c0196ec66d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:56.239695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934cb96-cb56-46dc-be7e-35c0196ec66d service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:56.254209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1221/2449] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:27:56.786713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f96454-7cfe-45b3-8c11-cc15539602a6 service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:27:56.800868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f96454-7cfe-45b3-8c11-cc15539602a6 service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:27:56.823114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb3141-5370-4c09-8437-61edf6bd19c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:27:56.824095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb3141-5370-4c09-8437-61edf6bd19c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:27:56.824495 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb3141-5370-4c09-8437-61edf6bd19c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:27:56.824686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb3141-5370-4c09-8437-61edf6bd19c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:27:56.827407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-e8bb3141-5370-4c09-8437-61edf6bd19c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:27:56.850257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb3141-5370-4c09-8437-61edf6bd19c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:27:56.852362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb3141-5370-4c09-8437-61edf6bd19c4 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:27:56.872952 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1229/2450] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:27:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:02.374280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934cb96-cb56-46dc-be7e-35c0196ec66d service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:02.375768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c934cb96-cb56-46dc-be7e-35c0196ec66d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:02.388301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934cb96-cb56-46dc-be7e-35c0196ec66d service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:02.393109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:02.394560 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:02.394560 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:02.394932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] PATH_INFO: `/v3/users/1c96cc61c2b84b3b81ed6c6738f9773e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:02.396003 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c96cc61c2b84b3b81ed6c6738f9773e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:02.397894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:02.622435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c96cc61c2b84b3b81ed6c6738f9773e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:02.629459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c96cc61c2b84b3b81ed6c6738f9773e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:02.710862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c96cc61c2b84b3b81ed6c6738f9773e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:02.711547 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c96cc61c2b84b3b81ed6c6738f9773e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:02.712175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-2d4b329c-d3ed-4237-ad92-b1df457bfc97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c96cc61c2b84b3b81ed6c6738f9773e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:02.732658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1222/2451] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:28:02 2022] DELETE /identity/v3/users/1c96cc61c2b84b3b81ed6c6738f9773e => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:02.963222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb3141-5370-4c09-8437-61edf6bd19c4 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:02.975348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb3141-5370-4c09-8437-61edf6bd19c4 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:03.035790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:03.036473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:03.038450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:03.038450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] PATH_INFO: `/v3/projects/fd509ff0932d40beb6ef2712e99a5fca` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.038450 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd509ff0932d40beb6ef2712e99a5fca)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:03.041739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:03.167801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd509ff0932d40beb6ef2712e99a5fca', 'request_id': 'req-b2b589b2-e337-4a87-b544-6d3eadfc7047'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:03.174010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd509ff0932d40beb6ef2712e99a5fca', 'request_id': 'req-b2b589b2-e337-4a87-b544-6d3eadfc7047'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:03.180060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd509ff0932d40beb6ef2712e99a5fca', 'request_id': 'req-b2b589b2-e337-4a87-b544-6d3eadfc7047'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:03.183565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:03.252538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] The token cache is being invalidate because project fd509ff0932d40beb6ef2712e99a5fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:28:03.252753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:03.253373 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b2b589b2-e337-4a87-b544-6d3eadfc7047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:03.255523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1230/2452] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:28:02 2022] DELETE /identity/v3/projects/fd509ff0932d40beb6ef2712e99a5fca => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:28:03.260590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68efda-d014-460d-ac17-fd272a1b9070 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:03.260884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68efda-d014-460d-ac17-fd272a1b9070 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:03.261166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68efda-d014-460d-ac17-fd272a1b9070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.280123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9d68efda-d014-460d-ac17-fd272a1b9070 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:03.281322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9d68efda-d014-460d-ac17-fd272a1b9070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:03.304115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:03.359837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1223/2453] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:03.363313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a12fd-a046-43ae-aa9e-ef7ef2efea86 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:03.364268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a12fd-a046-43ae-aa9e-ef7ef2efea86 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:03.364627 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a12fd-a046-43ae-aa9e-ef7ef2efea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.396722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8e2a12fd-a046-43ae-aa9e-ef7ef2efea86 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:03.400603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8e2a12fd-a046-43ae-aa9e-ef7ef2efea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:03.409476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1231/2454] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:03.414128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68efda-d014-460d-ac17-fd272a1b9070 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:03.424851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68efda-d014-460d-ac17-fd272a1b9070 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:03.431335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c475fa-35fc-443d-8cfa-c63b771ae75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:03.432420 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c475fa-35fc-443d-8cfa-c63b771ae75e admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:03.432772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c475fa-35fc-443d-8cfa-c63b771ae75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:03.433126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c475fa-35fc-443d-8cfa-c63b771ae75e admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.433955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c475fa-35fc-443d-8cfa-c63b771ae75e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:28:03.434324 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c475fa-35fc-443d-8cfa-c63b771ae75e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:03.436062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c475fa-35fc-443d-8cfa-c63b771ae75e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:03.450726 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1224/2455] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:28:03.455772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a12fd-a046-43ae-aa9e-ef7ef2efea86 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:03.465096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a12fd-a046-43ae-aa9e-ef7ef2efea86 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:03.469977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a392b00-e791-4207-8710-47fe14f5acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:03.470609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a392b00-e791-4207-8710-47fe14f5acfe admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:03.470897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a392b00-e791-4207-8710-47fe14f5acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:03.471204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a392b00-e791-4207-8710-47fe14f5acfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.471922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a392b00-e791-4207-8710-47fe14f5acfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:03.474792 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a392b00-e791-4207-8710-47fe14f5acfe admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:03.609978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1232/2456] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:28:03 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:28:03.614785 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c475fa-35fc-443d-8cfa-c63b771ae75e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:03.626592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c475fa-35fc-443d-8cfa-c63b771ae75e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:03.722352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51e6c0-0e49-46f1-88c5-b77bf9024cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:03.723821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51e6c0-0e49-46f1-88c5-b77bf9024cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:03.724946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51e6c0-0e49-46f1-88c5-b77bf9024cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:03.725225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51e6c0-0e49-46f1-88c5-b77bf9024cc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.726225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51e6c0-0e49-46f1-88c5-b77bf9024cc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:03.728339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51e6c0-0e49-46f1-88c5-b77bf9024cc0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:03.813605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1225/2457] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:28:03 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:28:03.818376 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a392b00-e791-4207-8710-47fe14f5acfe admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:03.828862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a392b00-e791-4207-8710-47fe14f5acfe admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:03.833700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0442c8-057c-4209-8c25-55f4af1a43ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:03.834427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0442c8-057c-4209-8c25-55f4af1a43ac admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:03.834787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0442c8-057c-4209-8c25-55f4af1a43ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:03.835129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0442c8-057c-4209-8c25-55f4af1a43ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.835978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0442c8-057c-4209-8c25-55f4af1a43ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:03.837651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0442c8-057c-4209-8c25-55f4af1a43ac admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:03.848572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1233/2458] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:28:03.852608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51e6c0-0e49-46f1-88c5-b77bf9024cc0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:03.866733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51e6c0-0e49-46f1-88c5-b77bf9024cc0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:03.871570 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c8eeb-700f-4b77-80b6-856b41e7fda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:03.872298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c8eeb-700f-4b77-80b6-856b41e7fda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:03.872653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c8eeb-700f-4b77-80b6-856b41e7fda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:03.873513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c8eeb-700f-4b77-80b6-856b41e7fda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.873691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c8eeb-700f-4b77-80b6-856b41e7fda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:03.875478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c8eeb-700f-4b77-80b6-856b41e7fda3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:03.924865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1226/2459] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:28:03.929343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0442c8-057c-4209-8c25-55f4af1a43ac admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:03.941042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0442c8-057c-4209-8c25-55f4af1a43ac admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:03.947919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04467f9f-825a-44f7-801c-86a4afef3339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:03.949048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04467f9f-825a-44f7-801c-86a4afef3339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:03.949547 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04467f9f-825a-44f7-801c-86a4afef3339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:03.950181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04467f9f-825a-44f7-801c-86a4afef3339 admin admin] PATH_INFO: `/v3/projects/d27d129141024af6955a9543e8c91b19/users/a2665de01f0446a8ba74bd6a161a0121/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:03.968123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04467f9f-825a-44f7-801c-86a4afef3339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27d129141024af6955a9543e8c91b19, user_id=a2665de01f0446a8ba74bd6a161a0121, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:03.970857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04467f9f-825a-44f7-801c-86a4afef3339 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:04.046632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1234/2460] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:28:03 2022] PUT /identity/v3/projects/d27d129141024af6955a9543e8c91b19/users/a2665de01f0446a8ba74bd6a161a0121/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:04.057197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932cb6-2a91-45ec-a932-32b15a3f1eb0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:04.057497 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932cb6-2a91-45ec-a932-32b15a3f1eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:04.057864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932cb6-2a91-45ec-a932-32b15a3f1eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:04.093467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-13932cb6-2a91-45ec-a932-32b15a3f1eb0 None None] MFA Rules not processed for user `a2665de01f0446a8ba74bd6a161a0121`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:04.095023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-13932cb6-2a91-45ec-a932-32b15a3f1eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:04.161276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1227/2461] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:04.172129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04467f9f-825a-44f7-801c-86a4afef3339 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:04.173862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-04467f9f-825a-44f7-801c-86a4afef3339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:04.273469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04467f9f-825a-44f7-801c-86a4afef3339 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:04.280256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199710f-353e-4bfe-8ba0-9c1c1267435b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:04.281021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199710f-353e-4bfe-8ba0-9c1c1267435b service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:04.281376 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199710f-353e-4bfe-8ba0-9c1c1267435b service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:04.281719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199710f-353e-4bfe-8ba0-9c1c1267435b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:04.299175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199710f-353e-4bfe-8ba0-9c1c1267435b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:04.299175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199710f-353e-4bfe-8ba0-9c1c1267435b service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:04.302486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1235/2462] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:06.453478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd027ce-db65-4523-8f64-28988cae136a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:06.455973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd027ce-db65-4523-8f64-28988cae136a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:06.455973 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd027ce-db65-4523-8f64-28988cae136a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:06.479023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-cfd027ce-db65-4523-8f64-28988cae136a None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:06.479023 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-cfd027ce-db65-4523-8f64-28988cae136a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:06.493889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:06.534351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1228/2463] 10.222.0.25 () {58 vars in 967 bytes} [Sat Aug 6 01:28:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:06.697054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199710f-353e-4bfe-8ba0-9c1c1267435b service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:06.702278 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c199710f-353e-4bfe-8ba0-9c1c1267435b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:06.765332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199710f-353e-4bfe-8ba0-9c1c1267435b service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:06.770746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7dba5-1825-49de-8e16-8671dfb41ad9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:06.771359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7dba5-1825-49de-8e16-8671dfb41ad9 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:06.771638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7dba5-1825-49de-8e16-8671dfb41ad9 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:06.771947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7dba5-1825-49de-8e16-8671dfb41ad9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:06.779937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7dba5-1825-49de-8e16-8671dfb41ad9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:06.781514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7dba5-1825-49de-8e16-8671dfb41ad9 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:06.790307 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1236/2464] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:28:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:28:07.600292 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd027ce-db65-4523-8f64-28988cae136a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:07.665339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd027ce-db65-4523-8f64-28988cae136a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:07.671272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:07.672026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:07.672311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:07.672590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] PATH_INFO: `/v3/users/a2665de01f0446a8ba74bd6a161a0121` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:07.673790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2665de01f0446a8ba74bd6a161a0121)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:07.675800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:07.820224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2665de01f0446a8ba74bd6a161a0121'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:07.826861 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2665de01f0446a8ba74bd6a161a0121'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:07.899924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2665de01f0446a8ba74bd6a161a0121'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:07.900897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2665de01f0446a8ba74bd6a161a0121'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:07.901915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2665de01f0446a8ba74bd6a161a0121'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:07.924163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1229/2465] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:28:07 2022] DELETE /identity/v3/users/a2665de01f0446a8ba74bd6a161a0121 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:28:07.929348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe106da-4f73-4185-99d7-653ddf89e085 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:07.930365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe106da-4f73-4185-99d7-653ddf89e085 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:07.930861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe106da-4f73-4185-99d7-653ddf89e085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:07.945793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:07.954250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ffe106da-4f73-4185-99d7-653ddf89e085 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:07.957183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ffe106da-4f73-4185-99d7-653ddf89e085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:08.029316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1237/2466] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:08.039163 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:08.040736 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:08.115484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:08.120701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e70d4-c9ae-40b3-b3e8-7abdbe42021d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:08.125088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5ce72-739e-49df-bcda-5d2883b53a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:08.125733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5ce72-739e-49df-bcda-5d2883b53a72 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:08.126019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5ce72-739e-49df-bcda-5d2883b53a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:08.126297 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5ce72-739e-49df-bcda-5d2883b53a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:08.136794 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5ce72-739e-49df-bcda-5d2883b53a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:08.138329 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5ce72-739e-49df-bcda-5d2883b53a72 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:08.149493 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1230/2467] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:28:08.603567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe106da-4f73-4185-99d7-653ddf89e085 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:08.605515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ffe106da-4f73-4185-99d7-653ddf89e085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:08.623121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe106da-4f73-4185-99d7-653ddf89e085 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:08.630730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:08.631664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:08.632187 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:08.632624 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] PATH_INFO: `/v3/projects/d27d129141024af6955a9543e8c91b19` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:08.634140 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27d129141024af6955a9543e8c91b19)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:08.636822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:08.658999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57949c92-f5f3-4bae-b426-30305666aa79 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:08.658999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57949c92-f5f3-4bae-b426-30305666aa79 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:08.658999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57949c92-f5f3-4bae-b426-30305666aa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:08.686068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-57949c92-f5f3-4bae-b426-30305666aa79 None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:08.687911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-57949c92-f5f3-4bae-b426-30305666aa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:08.748698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1231/2468] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:28:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:28:08.762378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57949c92-f5f3-4bae-b426-30305666aa79 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:08.767088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-57949c92-f5f3-4bae-b426-30305666aa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:08.779608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27d129141024af6955a9543e8c91b19', 'request_id': 'req-c13353e4-f6ca-4303-8db0-587b29e4300b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:08.786741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27d129141024af6955a9543e8c91b19', 'request_id': 'req-c13353e4-f6ca-4303-8db0-587b29e4300b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:08.794321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27d129141024af6955a9543e8c91b19', 'request_id': 'req-c13353e4-f6ca-4303-8db0-587b29e4300b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:08.854908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57949c92-f5f3-4bae-b426-30305666aa79 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:08.879586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] The token cache is being invalidate because project d27d129141024af6955a9543e8c91b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:28:08.879586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:08.879586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-c13353e4-f6ca-4303-8db0-587b29e4300b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:08.879586 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1238/2469] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:28:08 2022] DELETE /identity/v3/projects/d27d129141024af6955a9543e8c91b19 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:08.881124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28d3b5-5a24-4633-bab8-5c1c410b2cb1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:08.881404 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28d3b5-5a24-4633-bab8-5c1c410b2cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:08.881687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28d3b5-5a24-4633-bab8-5c1c410b2cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:08.937268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8a28d3b5-5a24-4633-bab8-5c1c410b2cb1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:08.937268 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8a28d3b5-5a24-4633-bab8-5c1c410b2cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:08.948864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36634d4e-99d4-427a-8382-e3bc3965629c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:08.949605 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36634d4e-99d4-427a-8382-e3bc3965629c service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:08.949853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36634d4e-99d4-427a-8382-e3bc3965629c service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:08.950132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36634d4e-99d4-427a-8382-e3bc3965629c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:08.952086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-36634d4e-99d4-427a-8382-e3bc3965629c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:09.006674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:09.022170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1239/2470] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:09.025568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0cfe-51b4-4d14-9bd6-159b4ea3d7a3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:09.025969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0cfe-51b4-4d14-9bd6-159b4ea3d7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:09.026352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0cfe-51b4-4d14-9bd6-159b4ea3d7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:09.029762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36634d4e-99d4-427a-8382-e3bc3965629c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:09.031928 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36634d4e-99d4-427a-8382-e3bc3965629c service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:09.042580 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1232/2471] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:28:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 01:28:09.058721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a56f0cfe-51b4-4d14-9bd6-159b4ea3d7a3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:09.060585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a56f0cfe-51b4-4d14-9bd6-159b4ea3d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:09.073036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1240/2472] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:09.077507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36634d4e-99d4-427a-8382-e3bc3965629c service swift] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:09.089066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36634d4e-99d4-427a-8382-e3bc3965629c service swift] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:09.093798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a318840-12fb-4bd8-9319-412cad629fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:09.094513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a318840-12fb-4bd8-9319-412cad629fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:09.094798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a318840-12fb-4bd8-9319-412cad629fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:09.095075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a318840-12fb-4bd8-9319-412cad629fe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:09.096150 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a318840-12fb-4bd8-9319-412cad629fe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:28:09.096267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a318840-12fb-4bd8-9319-412cad629fe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:09.097892 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a318840-12fb-4bd8-9319-412cad629fe2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:09.114879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1233/2473] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:28:09.116296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f0cfe-51b4-4d14-9bd6-159b4ea3d7a3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:09.126013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f0cfe-51b4-4d14-9bd6-159b4ea3d7a3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:09.131652 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7584e7-c6d9-447d-a5b4-5d2703b8639a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:09.132254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7584e7-c6d9-447d-a5b4-5d2703b8639a admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:09.132556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7584e7-c6d9-447d-a5b4-5d2703b8639a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:09.133250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7584e7-c6d9-447d-a5b4-5d2703b8639a admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:09.133513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7584e7-c6d9-447d-a5b4-5d2703b8639a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:09.135615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7584e7-c6d9-447d-a5b4-5d2703b8639a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:09.247809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1241/2474] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:28:09 2022] POST /identity/v3/projects => generated 353 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:28:09.255096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a318840-12fb-4bd8-9319-412cad629fe2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:09.273260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a318840-12fb-4bd8-9319-412cad629fe2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:09.351761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ba2e-41d1-4270-a9e4-9c7bbc82aba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:09.352442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ba2e-41d1-4270-a9e4-9c7bbc82aba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:09.353245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ba2e-41d1-4270-a9e4-9c7bbc82aba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:09.353245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ba2e-41d1-4270-a9e4-9c7bbc82aba1 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:09.353655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ba2e-41d1-4270-a9e4-9c7bbc82aba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:09.356316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ba2e-41d1-4270-a9e4-9c7bbc82aba1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:09.446927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1234/2475] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:28:09 2022] POST /identity/v3/users => generated 277 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:28:09.451033 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7584e7-c6d9-447d-a5b4-5d2703b8639a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:09.457114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:09.465418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7584e7-c6d9-447d-a5b4-5d2703b8639a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:09.477857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd2666-00e4-42bb-8250-50f4ceb4b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:09.482020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd2666-00e4-42bb-8250-50f4ceb4b24e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:09.482020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd2666-00e4-42bb-8250-50f4ceb4b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:09.482020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd2666-00e4-42bb-8250-50f4ceb4b24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:09.482399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd2666-00e4-42bb-8250-50f4ceb4b24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:09.484464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd2666-00e4-42bb-8250-50f4ceb4b24e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:09.499585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1242/2476] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:28:09.505000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ba2e-41d1-4270-a9e4-9c7bbc82aba1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:09.515096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ba2e-41d1-4270-a9e4-9c7bbc82aba1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:09.521137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552be48-70df-439f-8546-6b987559ac26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:09.521988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552be48-70df-439f-8546-6b987559ac26 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:09.522281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552be48-70df-439f-8546-6b987559ac26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:09.522550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552be48-70df-439f-8546-6b987559ac26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:09.524017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552be48-70df-439f-8546-6b987559ac26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:09.526859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552be48-70df-439f-8546-6b987559ac26 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:09.542874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1235/2477] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:28:09.545615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd2666-00e4-42bb-8250-50f4ceb4b24e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:09.561439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd2666-00e4-42bb-8250-50f4ceb4b24e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:09.567609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63509880-ee0c-43af-b1d6-25a76c229be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:09.568492 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63509880-ee0c-43af-b1d6-25a76c229be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:09.568789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63509880-ee0c-43af-b1d6-25a76c229be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:09.569102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63509880-ee0c-43af-b1d6-25a76c229be8 admin admin] PATH_INFO: `/v3/projects/d4b18406dda647afbfcc16e2214766ba/users/c04d04c623ba49b4a950cd1b8da02124/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:09.589734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63509880-ee0c-43af-b1d6-25a76c229be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b18406dda647afbfcc16e2214766ba, user_id=c04d04c623ba49b4a950cd1b8da02124, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:09.592112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552be48-70df-439f-8546-6b987559ac26 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:09.593562 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63509880-ee0c-43af-b1d6-25a76c229be8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:09.594458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8552be48-70df-439f-8546-6b987559ac26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:09.696312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1243/2478] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:28:09 2022] PUT /identity/v3/projects/d4b18406dda647afbfcc16e2214766ba/users/c04d04c623ba49b4a950cd1b8da02124/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:28:09.698425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7f05d-2ae1-47d6-bb2f-fd43658d5a8a None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:09.699183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7f05d-2ae1-47d6-bb2f-fd43658d5a8a None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:09.699495 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7f05d-2ae1-47d6-bb2f-fd43658d5a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:09.718096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552be48-70df-439f-8546-6b987559ac26 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:09.775949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-57a7f05d-2ae1-47d6-bb2f-fd43658d5a8a None None] MFA Rules not processed for user `c04d04c623ba49b4a950cd1b8da02124`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:09.777196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-57a7f05d-2ae1-47d6-bb2f-fd43658d5a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:09.830296 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:09.864879 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccbc73-2a89-4cc4-a159-ed919d16a681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:09.865757 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccbc73-2a89-4cc4-a159-ed919d16a681 service nova] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:09.866223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccbc73-2a89-4cc4-a159-ed919d16a681 service nova] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:09.866651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccbc73-2a89-4cc4-a159-ed919d16a681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:09.868851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-94ccbc73-2a89-4cc4-a159-ed919d16a681 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:09.886266 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccbc73-2a89-4cc4-a159-ed919d16a681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:09.889637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccbc73-2a89-4cc4-a159-ed919d16a681 service nova] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:09.908410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1236/2479] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:28:09.926651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1244/2480] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:09 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:28:09.932425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccbc73-2a89-4cc4-a159-ed919d16a681 service nova] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:10.105053 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccbc73-2a89-4cc4-a159-ed919d16a681 service nova] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:10.112382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdfbf0-d662-433a-8c57-67125c446d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:10.113631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdfbf0-d662-433a-8c57-67125c446d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:10.114167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdfbf0-d662-433a-8c57-67125c446d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:10.114669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdfbf0-d662-433a-8c57-67125c446d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:10.116362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdfbf0-d662-433a-8c57-67125c446d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:10.120084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdfbf0-d662-433a-8c57-67125c446d03 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:10.229831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1237/2481] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:28:09 2022] POST /identity/v3/projects => generated 357 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:28:10.234231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7f05d-2ae1-47d6-bb2f-fd43658d5a8a None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:10.255660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7f05d-2ae1-47d6-bb2f-fd43658d5a8a None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:10.400751 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd99e9-c289-4c63-bc66-5dd0ac43aad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:10.401397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd99e9-c289-4c63-bc66-5dd0ac43aad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:10.401689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd99e9-c289-4c63-bc66-5dd0ac43aad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:10.402042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd99e9-c289-4c63-bc66-5dd0ac43aad4 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:10.402731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd99e9-c289-4c63-bc66-5dd0ac43aad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:10.405407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd99e9-c289-4c63-bc66-5dd0ac43aad4 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:10.506641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1245/2482] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:28:10 2022] POST /identity/v3/users => generated 285 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:28:10.511612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdfbf0-d662-433a-8c57-67125c446d03 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:10.527193 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdfbf0-d662-433a-8c57-67125c446d03 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:10.539763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80def542-998c-4c49-91b1-1b49f6980465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:10.540506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def542-998c-4c49-91b1-1b49f6980465 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:10.540860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def542-998c-4c49-91b1-1b49f6980465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:10.541140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80def542-998c-4c49-91b1-1b49f6980465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:10.541922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80def542-998c-4c49-91b1-1b49f6980465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:10.547922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80def542-998c-4c49-91b1-1b49f6980465 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:10.568032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1238/2483] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:28:10.572138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd99e9-c289-4c63-bc66-5dd0ac43aad4 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:10.598946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd99e9-c289-4c63-bc66-5dd0ac43aad4 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:10.604120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b48efe-b5f5-40e3-af4a-58d2699a832b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:10.606103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b48efe-b5f5-40e3-af4a-58d2699a832b admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:10.606103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b48efe-b5f5-40e3-af4a-58d2699a832b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:10.606103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b48efe-b5f5-40e3-af4a-58d2699a832b admin admin] PATH_INFO: `/v3/projects/840fd6b1d8c04e27a65466babec9693d/users/4dc6b0d7e70640dc87cc4c87cf6cb08a/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:10.636331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b48efe-b5f5-40e3-af4a-58d2699a832b admin admin] RBAC: Authorizing `identity:create_grant(project_id=840fd6b1d8c04e27a65466babec9693d, user_id=4dc6b0d7e70640dc87cc4c87cf6cb08a, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:10.639753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b48efe-b5f5-40e3-af4a-58d2699a832b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:10.708349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1246/2484] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:28:10 2022] PUT /identity/v3/projects/840fd6b1d8c04e27a65466babec9693d/users/4dc6b0d7e70640dc87cc4c87cf6cb08a/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:10.712961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80def542-998c-4c49-91b1-1b49f6980465 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:10.734205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80def542-998c-4c49-91b1-1b49f6980465 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:10.860907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79493263-4526-447c-bdd8-f2eeaf663235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:10.861788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79493263-4526-447c-bdd8-f2eeaf663235 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:10.862241 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79493263-4526-447c-bdd8-f2eeaf663235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:10.862680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79493263-4526-447c-bdd8-f2eeaf663235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:10.864451 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79493263-4526-447c-bdd8-f2eeaf663235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:10.867268 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79493263-4526-447c-bdd8-f2eeaf663235 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:10.902475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1239/2485] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:28:10.903734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b48efe-b5f5-40e3-af4a-58d2699a832b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:10.924020 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b48efe-b5f5-40e3-af4a-58d2699a832b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:10.938883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed418fe7-b389-42b5-a449-4b4aa775ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:10.939748 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed418fe7-b389-42b5-a449-4b4aa775ecb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:10.940187 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed418fe7-b389-42b5-a449-4b4aa775ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:10.940587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed418fe7-b389-42b5-a449-4b4aa775ecb0 admin admin] PATH_INFO: `/v3/projects/840fd6b1d8c04e27a65466babec9693d/users/4dc6b0d7e70640dc87cc4c87cf6cb08a/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:10.943520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed418fe7-b389-42b5-a449-4b4aa775ecb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840fd6b1d8c04e27a65466babec9693d, user_id=4dc6b0d7e70640dc87cc4c87cf6cb08a, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:10.947081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed418fe7-b389-42b5-a449-4b4aa775ecb0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:10.969942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1247/2486] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:28:10 2022] PUT /identity/v3/projects/840fd6b1d8c04e27a65466babec9693d/users/4dc6b0d7e70640dc87cc4c87cf6cb08a/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:10.986105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57b8b5-d168-4105-ba67-8b266e1bf40d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:10.986331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57b8b5-d168-4105-ba67-8b266e1bf40d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:10.986621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57b8b5-d168-4105-ba67-8b266e1bf40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:11.079899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-2e57b8b5-d168-4105-ba67-8b266e1bf40d None None] MFA Rules not processed for user `4dc6b0d7e70640dc87cc4c87cf6cb08a`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:11.081566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2e57b8b5-d168-4105-ba67-8b266e1bf40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:11.148569 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:11.278177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1240/2487] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:10 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:11.287820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed418fe7-b389-42b5-a449-4b4aa775ecb0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:11.291250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ed418fe7-b389-42b5-a449-4b4aa775ecb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:11.319772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:11.434090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed418fe7-b389-42b5-a449-4b4aa775ecb0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:11.442894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15fff2-dc33-4eba-ae08-477ca50798f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:11.443700 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15fff2-dc33-4eba-ae08-477ca50798f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:11.444131 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15fff2-dc33-4eba-ae08-477ca50798f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:11.444928 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15fff2-dc33-4eba-ae08-477ca50798f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:11.469540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15fff2-dc33-4eba-ae08-477ca50798f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:11.473834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15fff2-dc33-4eba-ae08-477ca50798f1 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:11.506919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1248/2488] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:11 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:11.797224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57b8b5-d168-4105-ba67-8b266e1bf40d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:11.818047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57b8b5-d168-4105-ba67-8b266e1bf40d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:11.824556 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a1a98-8685-4686-a7f0-27bda77c2590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:11.825212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a1a98-8685-4686-a7f0-27bda77c2590 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:11.825492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a1a98-8685-4686-a7f0-27bda77c2590 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:11.825767 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a1a98-8685-4686-a7f0-27bda77c2590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:12.018185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a1a98-8685-4686-a7f0-27bda77c2590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:12.025123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a1a98-8685-4686-a7f0-27bda77c2590 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:12.064469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1241/2489] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:11 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:17.646768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582994-48b8-451f-bd63-23c66c3b8c95 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:17.647346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582994-48b8-451f-bd63-23c66c3b8c95 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:17.647346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582994-48b8-451f-bd63-23c66c3b8c95 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:17.648679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1249/2490] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:28:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:28:17.661591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b256a7-92e6-44c4-853e-ccfc62a714e2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:17.662074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b256a7-92e6-44c4-853e-ccfc62a714e2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:17.662187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b256a7-92e6-44c4-853e-ccfc62a714e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:17.725112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d5b256a7-92e6-44c4-853e-ccfc62a714e2 None None] MFA Rules not processed for user `2c14fb133a0f419390fb0d00c600bdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:17.730485 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d5b256a7-92e6-44c4-853e-ccfc62a714e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:17.866224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1242/2491] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:28:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:17.884200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582994-48b8-451f-bd63-23c66c3b8c95 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:17.896556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-88582994-48b8-451f-bd63-23c66c3b8c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:17.913899 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582994-48b8-451f-bd63-23c66c3b8c95 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:17.918390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c145183-5821-4fff-973a-9d4b6e32ee64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:17.919109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c145183-5821-4fff-973a-9d4b6e32ee64 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:17.919658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c145183-5821-4fff-973a-9d4b6e32ee64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:17.920203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c145183-5821-4fff-973a-9d4b6e32ee64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:17.928481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c145183-5821-4fff-973a-9d4b6e32ee64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:17.930014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c145183-5821-4fff-973a-9d4b6e32ee64 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:17.948123 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1250/2492] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:18.111049 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273cb52-6252-468f-9dda-80cc737779ed None None] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:18.111339 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273cb52-6252-468f-9dda-80cc737779ed None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:18.111610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273cb52-6252-468f-9dda-80cc737779ed None None] PATH_INFO: `/` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:18.113934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1243/2493] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:28:18.119460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdb045-879c-4388-9289-9014a44fecc8 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:18.119744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdb045-879c-4388-9289-9014a44fecc8 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:18.120138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdb045-879c-4388-9289-9014a44fecc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:18.198563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a1cdb045-879c-4388-9289-9014a44fecc8 None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:18.198563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a1cdb045-879c-4388-9289-9014a44fecc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:18.286598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1251/2494] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:28:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:18.310793 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273cb52-6252-468f-9dda-80cc737779ed None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:18.328671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273cb52-6252-468f-9dda-80cc737779ed None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:18.333882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163feee9-f749-464d-b397-374abf4654db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:18.341365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163feee9-f749-464d-b397-374abf4654db service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:18.341365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163feee9-f749-464d-b397-374abf4654db service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:18.341365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163feee9-f749-464d-b397-374abf4654db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:18.348808 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163feee9-f749-464d-b397-374abf4654db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:18.350549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163feee9-f749-464d-b397-374abf4654db service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:18.365916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1244/2495] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:19.996102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdb045-879c-4388-9289-9014a44fecc8 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:20.000183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a1cdb045-879c-4388-9289-9014a44fecc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:20.098543 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdb045-879c-4388-9289-9014a44fecc8 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:20.113567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ac4af-df71-4644-8af4-b2578f34062f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:20.114384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ac4af-df71-4644-8af4-b2578f34062f service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:20.115820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ac4af-df71-4644-8af4-b2578f34062f service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:20.116263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ac4af-df71-4644-8af4-b2578f34062f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:20.118827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7b2ac4af-df71-4644-8af4-b2578f34062f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:20.129798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163feee9-f749-464d-b397-374abf4654db service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:20.131253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-163feee9-f749-464d-b397-374abf4654db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:20.146710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ac4af-df71-4644-8af4-b2578f34062f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:20.149332 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ac4af-df71-4644-8af4-b2578f34062f service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:20.173000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1252/2496] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:20.253599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:20.260470 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163feee9-f749-464d-b397-374abf4654db service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:20.267027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:20.272233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:20.272233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:20.272233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] PATH_INFO: `/v3/users/ee469b9be6bf4ef192560551e817b385` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:20.272233 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee469b9be6bf4ef192560551e817b385)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:20.273108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:20.497076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee469b9be6bf4ef192560551e817b385'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:20.512863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee469b9be6bf4ef192560551e817b385'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:20.605029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee469b9be6bf4ef192560551e817b385'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:20.605272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee469b9be6bf4ef192560551e817b385'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:20.606027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee469b9be6bf4ef192560551e817b385'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:20.646201 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1245/2497] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:28:20 2022] DELETE /identity/v3/users/ee469b9be6bf4ef192560551e817b385 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:21.023387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ac4af-df71-4644-8af4-b2578f34062f service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:21.043477 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ac4af-df71-4644-8af4-b2578f34062f service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:21.080109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:21.174121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:21.174854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:21.175146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:21.175488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] PATH_INFO: `/v3/projects/019c42b12f444a35a1617fc7383c212c` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:21.177465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=019c42b12f444a35a1617fc7383c212c)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:21.179421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:21.351745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '019c42b12f444a35a1617fc7383c212c', 'request_id': 'req-61c7a11e-063e-4092-905b-6c996b9097d0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:21.373948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '019c42b12f444a35a1617fc7383c212c', 'request_id': 'req-61c7a11e-063e-4092-905b-6c996b9097d0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:21.391265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '019c42b12f444a35a1617fc7383c212c', 'request_id': 'req-61c7a11e-063e-4092-905b-6c996b9097d0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:21.515876 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] The token cache is being invalidate because project 019c42b12f444a35a1617fc7383c212c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:28:21.516327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:21.517053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:21.522510 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1253/2498] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:28:21 2022] DELETE /identity/v3/projects/019c42b12f444a35a1617fc7383c212c => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:21.523812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:21.525966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:21.548502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed054248-9360-483d-afdc-26f2ee047693 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:21.672084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:21.672825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:21.673115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:21.673543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] PATH_INFO: `/v3/users/79c5895434dd45469057b70a24c34a35` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:21.676852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c5895434dd45469057b70a24c34a35)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:21.680782 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:21.934417 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c5895434dd45469057b70a24c34a35'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:21.951048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c5895434dd45469057b70a24c34a35'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:22.066776 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c5895434dd45469057b70a24c34a35'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:22.068623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c5895434dd45469057b70a24c34a35'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:22.069250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-94f123c6-4b9a-49bf-8eb6-f7e76996b4c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c5895434dd45469057b70a24c34a35'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:22.089592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:22.097377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1246/2499] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:28:21 2022] DELETE /identity/v3/users/79c5895434dd45469057b70a24c34a35 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:23.047528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:23.049370 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:23.076915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7a11e-063e-4092-905b-6c996b9097d0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:23.201998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:23.204970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:23.205567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:23.208440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] PATH_INFO: `/v3/projects/bbe0d42b9127436ab41248d9b6e174a9` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:23.210018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbe0d42b9127436ab41248d9b6e174a9)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:23.212333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:23.391327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe0d42b9127436ab41248d9b6e174a9', 'request_id': 'req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:23.405153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:23.412310 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe0d42b9127436ab41248d9b6e174a9', 'request_id': 'req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:23.428248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe0d42b9127436ab41248d9b6e174a9', 'request_id': 'req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:23.520578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] The token cache is being invalidate because project bbe0d42b9127436ab41248d9b6e174a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:28:23.520578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:23.524418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ca9319ff-57ee-46d6-8f6e-f28615c2dbe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:23.540715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1254/2500] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:28:23 2022] DELETE /identity/v3/projects/bbe0d42b9127436ab41248d9b6e174a9 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:25.384552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a135e-e910-45fd-b6af-3959450a4348 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:25.385034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a135e-e910-45fd-b6af-3959450a4348 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:25.385358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a135e-e910-45fd-b6af-3959450a4348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:25.404935 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-273a135e-e910-45fd-b6af-3959450a4348 None None] MFA Rules not processed for user `42047fe93eb24adbbc24844a75ed21f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:25.406224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-273a135e-e910-45fd-b6af-3959450a4348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:25.464864 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1247/2501] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:31.477735 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31fe49-a21b-4ab0-b836-79eebce03f09 None None] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:31.478312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31fe49-a21b-4ab0-b836-79eebce03f09 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:31.478428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31fe49-a21b-4ab0-b836-79eebce03f09 None None] PATH_INFO: `/` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:31.479842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1255/2502] 10.222.0.25 () {54 vars in 921 bytes} [Sat Aug 6 01:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 01:28:31.485006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93fb53-08ea-4d39-928a-115ca63a6156 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:31.485625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93fb53-08ea-4d39-928a-115ca63a6156 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:31.485957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93fb53-08ea-4d39-928a-115ca63a6156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:31.506721 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0e93fb53-08ea-4d39-928a-115ca63a6156 None None] MFA Rules not processed for user `db1b205d165543d0a3a69a2db6af5d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:31.508360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0e93fb53-08ea-4d39-928a-115ca63a6156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:31.649229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1248/2503] 10.222.0.25 () {58 vars in 1018 bytes} [Sat Aug 6 01:28:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:28:31.663955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31fe49-a21b-4ab0-b836-79eebce03f09 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:31.666165 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-af31fe49-a21b-4ab0-b836-79eebce03f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:31.735432 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31fe49-a21b-4ab0-b836-79eebce03f09 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:31.739967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9daa8-2554-44e1-af0f-2a3a1c5f6501 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a234a96eccd4f38b0213d7d0461ad24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:31.740604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9daa8-2554-44e1-af0f-2a3a1c5f6501 service glance] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:31.741272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9daa8-2554-44e1-af0f-2a3a1c5f6501 service glance] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:31.741570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9daa8-2554-44e1-af0f-2a3a1c5f6501 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:31.753024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9daa8-2554-44e1-af0f-2a3a1c5f6501 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:31.755171 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9daa8-2554-44e1-af0f-2a3a1c5f6501 service glance] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:31.775440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1256/2504] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:33.490701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93fb53-08ea-4d39-928a-115ca63a6156 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:33.492178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0e93fb53-08ea-4d39-928a-115ca63a6156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:33.561747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93fb53-08ea-4d39-928a-115ca63a6156 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:33.566946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:33.567595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:33.567910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:33.568190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] PATH_INFO: `/v3/users/c04d04c623ba49b4a950cd1b8da02124` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:33.569379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c04d04c623ba49b4a950cd1b8da02124)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:33.571354 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:33.796087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:33.807186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c04d04c623ba49b4a950cd1b8da02124'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:33.814576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c04d04c623ba49b4a950cd1b8da02124'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:33.876506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c04d04c623ba49b4a950cd1b8da02124'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:33.877158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c04d04c623ba49b4a950cd1b8da02124'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:33.877932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c04d04c623ba49b4a950cd1b8da02124'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:33.900745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1249/2505] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:28:33 2022] DELETE /identity/v3/users/c04d04c623ba49b4a950cd1b8da02124 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:33.905871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0f8c0-ef42-414a-8723-875a13f67611 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:33.906396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0f8c0-ef42-414a-8723-875a13f67611 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:33.909157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0f8c0-ef42-414a-8723-875a13f67611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:33.941540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8ce0f8c0-ef42-414a-8723-875a13f67611 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:33.943046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8ce0f8c0-ef42-414a-8723-875a13f67611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:34.018641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1257/2506] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:34.044290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:34.046487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:34.125144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fea32d-d871-4593-ad0a-dffac9ad4173 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:34.129445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cde0ae-3dfa-4955-8fb6-7c5668d02c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:34.130329 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde0ae-3dfa-4955-8fb6-7c5668d02c4f service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:34.130806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde0ae-3dfa-4955-8fb6-7c5668d02c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:34.131272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde0ae-3dfa-4955-8fb6-7c5668d02c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:34.141433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cde0ae-3dfa-4955-8fb6-7c5668d02c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:34.142869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cde0ae-3dfa-4955-8fb6-7c5668d02c4f service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:34.153132 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1250/2507] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:34.580160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0f8c0-ef42-414a-8723-875a13f67611 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:34.581844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8ce0f8c0-ef42-414a-8723-875a13f67611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:34.596656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0f8c0-ef42-414a-8723-875a13f67611 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:34.601682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:34.602371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:34.602654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:34.607110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] PATH_INFO: `/v3/projects/d4b18406dda647afbfcc16e2214766ba` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:34.607110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4b18406dda647afbfcc16e2214766ba)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:34.607110 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:34.721230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4b18406dda647afbfcc16e2214766ba', 'request_id': 'req-0584b542-3c22-430d-90a7-1dd11d911005'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:34.727357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4b18406dda647afbfcc16e2214766ba', 'request_id': 'req-0584b542-3c22-430d-90a7-1dd11d911005'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:34.733144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4b18406dda647afbfcc16e2214766ba', 'request_id': 'req-0584b542-3c22-430d-90a7-1dd11d911005'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:34.807753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] The token cache is being invalidate because project d4b18406dda647afbfcc16e2214766ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:28:34.808079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:34.808723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0584b542-3c22-430d-90a7-1dd11d911005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:34.810512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1258/2508] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:28:34 2022] DELETE /identity/v3/projects/d4b18406dda647afbfcc16e2214766ba => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:34.814526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cde0ae-3dfa-4955-8fb6-7c5668d02c4f service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:34.824918 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cde0ae-3dfa-4955-8fb6-7c5668d02c4f service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:34.883800 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:34.884484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:34.884762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:34.885036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] PATH_INFO: `/v3/users/4dc6b0d7e70640dc87cc4c87cf6cb08a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:34.886179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc6b0d7e70640dc87cc4c87cf6cb08a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:34.888122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:35.307804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc6b0d7e70640dc87cc4c87cf6cb08a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:35.314698 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc6b0d7e70640dc87cc4c87cf6cb08a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:35.316534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:35.422761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc6b0d7e70640dc87cc4c87cf6cb08a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:35.423521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc6b0d7e70640dc87cc4c87cf6cb08a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:35.424166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc6b0d7e70640dc87cc4c87cf6cb08a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:35.444312 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1251/2509] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:28:34 2022] DELETE /identity/v3/users/4dc6b0d7e70640dc87cc4c87cf6cb08a => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:35.745973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200c98e-1085-492f-bd0b-5ae26230c690 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:35.746486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200c98e-1085-492f-bd0b-5ae26230c690 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:35.746612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200c98e-1085-492f-bd0b-5ae26230c690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:35.761540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:35.767300 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-9200c98e-1085-492f-bd0b-5ae26230c690 None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:35.768554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9200c98e-1085-492f-bd0b-5ae26230c690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:35.826169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1259/2510] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:28:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:35.839807 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:35.841271 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:35.909923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eb61f-31a4-41db-bd14-a0c154284c98 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:35.915598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aca479-cb55-44fa-a4c4-20584b76c1cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:35.916268 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aca479-cb55-44fa-a4c4-20584b76c1cb service swift] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:35.916527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aca479-cb55-44fa-a4c4-20584b76c1cb service swift] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:35.916798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aca479-cb55-44fa-a4c4-20584b76c1cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:35.925880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aca479-cb55-44fa-a4c4-20584b76c1cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:35.927578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aca479-cb55-44fa-a4c4-20584b76c1cb service swift] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:35.937187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1252/2511] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:28:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:28:35.946128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200c98e-1085-492f-bd0b-5ae26230c690 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:35.947868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9200c98e-1085-492f-bd0b-5ae26230c690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:36.015295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200c98e-1085-492f-bd0b-5ae26230c690 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:36.020207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:36.023822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:36.023822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:36.023822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] PATH_INFO: `/v3/projects/840fd6b1d8c04e27a65466babec9693d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:36.023822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=840fd6b1d8c04e27a65466babec9693d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:36.024620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:36.155948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840fd6b1d8c04e27a65466babec9693d', 'request_id': 'req-aab76f8a-8814-4085-860f-fd7afc4fb701'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:36.170325 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840fd6b1d8c04e27a65466babec9693d', 'request_id': 'req-aab76f8a-8814-4085-860f-fd7afc4fb701'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:36.178580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840fd6b1d8c04e27a65466babec9693d', 'request_id': 'req-aab76f8a-8814-4085-860f-fd7afc4fb701'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:36.302545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] The token cache is being invalidate because project 840fd6b1d8c04e27a65466babec9693d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:28:36.302801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:36.303650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-aab76f8a-8814-4085-860f-fd7afc4fb701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:28:36.305327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1260/2512] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:28:35 2022] DELETE /identity/v3/projects/840fd6b1d8c04e27a65466babec9693d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:36.310585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00961d1-067b-46e3-b4ef-70b381d18a98 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:36.310944 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00961d1-067b-46e3-b4ef-70b381d18a98 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:36.311478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00961d1-067b-46e3-b4ef-70b381d18a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:36.339972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d00961d1-067b-46e3-b4ef-70b381d18a98 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:36.341884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d00961d1-067b-46e3-b4ef-70b381d18a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:36.433290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1253/2513] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:36.441513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eceb85-4fec-46dc-90ed-e489f3f616ca None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:36.441832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eceb85-4fec-46dc-90ed-e489f3f616ca None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:36.442113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eceb85-4fec-46dc-90ed-e489f3f616ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:36.467954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-52eceb85-4fec-46dc-90ed-e489f3f616ca None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:36.469484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-52eceb85-4fec-46dc-90ed-e489f3f616ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:36.482278 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00961d1-067b-46e3-b4ef-70b381d18a98 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:36.487364 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1261/2514] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:36.492988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00961d1-067b-46e3-b4ef-70b381d18a98 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:36.500039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01566d0-42b6-4b9c-939b-b8cc8c72fc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:36.500744 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01566d0-42b6-4b9c-939b-b8cc8c72fc21 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:36.501118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01566d0-42b6-4b9c-939b-b8cc8c72fc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:36.501503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01566d0-42b6-4b9c-939b-b8cc8c72fc21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:36.502360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01566d0-42b6-4b9c-939b-b8cc8c72fc21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:28:36.502827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01566d0-42b6-4b9c-939b-b8cc8c72fc21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:36.506825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01566d0-42b6-4b9c-939b-b8cc8c72fc21 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:36.527975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1254/2515] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:28:36.529082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eceb85-4fec-46dc-90ed-e489f3f616ca None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:36.540714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eceb85-4fec-46dc-90ed-e489f3f616ca None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:36.548580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddbf9b-c56b-4c60-8464-fd8315b2da88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:36.549323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddbf9b-c56b-4c60-8464-fd8315b2da88 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:36.549658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddbf9b-c56b-4c60-8464-fd8315b2da88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:36.550026 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddbf9b-c56b-4c60-8464-fd8315b2da88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:36.550863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eddbf9b-c56b-4c60-8464-fd8315b2da88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:36.553567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eddbf9b-c56b-4c60-8464-fd8315b2da88 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:36.663304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1262/2516] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:28:36 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:28:36.667019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01566d0-42b6-4b9c-939b-b8cc8c72fc21 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:36.677816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01566d0-42b6-4b9c-939b-b8cc8c72fc21 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:36.752377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:36.769573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7393ea1-06d3-4339-b03d-830fa23edcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:36.773779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7393ea1-06d3-4339-b03d-830fa23edcb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:36.774660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7393ea1-06d3-4339-b03d-830fa23edcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:36.775051 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7393ea1-06d3-4339-b03d-830fa23edcb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:36.776689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7393ea1-06d3-4339-b03d-830fa23edcb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:36.780016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7393ea1-06d3-4339-b03d-830fa23edcb2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:36.889655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1255/2517] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:28:36 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:28:36.899076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddbf9b-c56b-4c60-8464-fd8315b2da88 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:36.913409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddbf9b-c56b-4c60-8464-fd8315b2da88 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:36.921208 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a066710-a4e8-4df8-a144-f08869b54047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:36.922329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a066710-a4e8-4df8-a144-f08869b54047 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:36.923334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a066710-a4e8-4df8-a144-f08869b54047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:36.923334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a066710-a4e8-4df8-a144-f08869b54047 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:36.923828 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a066710-a4e8-4df8-a144-f08869b54047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:36.927753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a066710-a4e8-4df8-a144-f08869b54047 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:36.937629 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1263/2518] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:28:36.941835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7393ea1-06d3-4339-b03d-830fa23edcb2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:36.964643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7393ea1-06d3-4339-b03d-830fa23edcb2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:36.974716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e515c-a8fe-409a-8e29-0c2487afbc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:36.975851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e515c-a8fe-409a-8e29-0c2487afbc70 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:36.979458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e515c-a8fe-409a-8e29-0c2487afbc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:36.979458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e515c-a8fe-409a-8e29-0c2487afbc70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:36.979458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e515c-a8fe-409a-8e29-0c2487afbc70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:36.979458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134e515c-a8fe-409a-8e29-0c2487afbc70 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:36.992969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1256/2519] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:28:36.997442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a066710-a4e8-4df8-a144-f08869b54047 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:37.003011 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:37.013018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a066710-a4e8-4df8-a144-f08869b54047 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:37.018956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ea142-05f2-4090-953f-120650b41ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:37.023846 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea142-05f2-4090-953f-120650b41ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:37.024347 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea142-05f2-4090-953f-120650b41ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:37.024831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea142-05f2-4090-953f-120650b41ca5 admin admin] PATH_INFO: `/v3/projects/19bad7d46cf84b83bfd77b063744298d/users/a159c9b1203544c8942c0e7d15eafd3d/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:37.050833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ea142-05f2-4090-953f-120650b41ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19bad7d46cf84b83bfd77b063744298d, user_id=a159c9b1203544c8942c0e7d15eafd3d, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:37.053812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ea142-05f2-4090-953f-120650b41ca5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:37.131765 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1264/2520] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:28:36 2022] PUT /identity/v3/projects/19bad7d46cf84b83bfd77b063744298d/users/a159c9b1203544c8942c0e7d15eafd3d/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:37.148502 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08942f24-1038-42af-87ef-0220fd20ef1c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:37.148996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08942f24-1038-42af-87ef-0220fd20ef1c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:37.149401 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08942f24-1038-42af-87ef-0220fd20ef1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:37.191920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-08942f24-1038-42af-87ef-0220fd20ef1c None None] MFA Rules not processed for user `a159c9b1203544c8942c0e7d15eafd3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:37.193909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-08942f24-1038-42af-87ef-0220fd20ef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:37.282934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1257/2521] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:37.288757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ea142-05f2-4090-953f-120650b41ca5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:37.366563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ea142-05f2-4090-953f-120650b41ca5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:37.372516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5410f8-13c3-4f67-9704-be892a5e9ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:37.373235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5410f8-13c3-4f67-9704-be892a5e9ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:37.373659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5410f8-13c3-4f67-9704-be892a5e9ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:37.373950 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5410f8-13c3-4f67-9704-be892a5e9ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:37.374692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5410f8-13c3-4f67-9704-be892a5e9ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:37.377007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5410f8-13c3-4f67-9704-be892a5e9ce3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:37.467169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1265/2522] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:28:37 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:28:37.472395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08942f24-1038-42af-87ef-0220fd20ef1c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:37.484836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08942f24-1038-42af-87ef-0220fd20ef1c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:37.559126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baceb36-8c1c-4a92-aa55-c20b5730d4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:37.559898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baceb36-8c1c-4a92-aa55-c20b5730d4f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:37.560264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baceb36-8c1c-4a92-aa55-c20b5730d4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:37.560733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baceb36-8c1c-4a92-aa55-c20b5730d4f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:37.561510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baceb36-8c1c-4a92-aa55-c20b5730d4f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:37.563548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baceb36-8c1c-4a92-aa55-c20b5730d4f8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:37.648141 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1258/2523] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:28:37 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:28:37.652878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5410f8-13c3-4f67-9704-be892a5e9ce3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:37.665323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5410f8-13c3-4f67-9704-be892a5e9ce3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:37.670650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966f65b-461f-42d5-8f8c-7304b1719bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:37.671298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966f65b-461f-42d5-8f8c-7304b1719bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:37.671657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966f65b-461f-42d5-8f8c-7304b1719bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:37.672047 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966f65b-461f-42d5-8f8c-7304b1719bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:37.672834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966f65b-461f-42d5-8f8c-7304b1719bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:37.674577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966f65b-461f-42d5-8f8c-7304b1719bc3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:37.687953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1266/2524] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:28:37.692291 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baceb36-8c1c-4a92-aa55-c20b5730d4f8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:37.705226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baceb36-8c1c-4a92-aa55-c20b5730d4f8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:37.712190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189665ef-8b25-43c5-bde0-16233c6ab5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:37.713097 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189665ef-8b25-43c5-bde0-16233c6ab5ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:37.713568 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189665ef-8b25-43c5-bde0-16233c6ab5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:37.713949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189665ef-8b25-43c5-bde0-16233c6ab5ca admin admin] PATH_INFO: `/v3/projects/d3a7edc4db5c44f68377d44e9f20d6bb/users/eea90c1514254bf68bbde782cebbb32e/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:37.733154 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189665ef-8b25-43c5-bde0-16233c6ab5ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a7edc4db5c44f68377d44e9f20d6bb, user_id=eea90c1514254bf68bbde782cebbb32e, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:37.736334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189665ef-8b25-43c5-bde0-16233c6ab5ca admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:37.807827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1259/2525] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:28:37 2022] PUT /identity/v3/projects/d3a7edc4db5c44f68377d44e9f20d6bb/users/eea90c1514254bf68bbde782cebbb32e/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:37.811923 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966f65b-461f-42d5-8f8c-7304b1719bc3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:37.822203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966f65b-461f-42d5-8f8c-7304b1719bc3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:37.939711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab3692-19f9-40b1-b98b-68ba81f88402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:37.940890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3692-19f9-40b1-b98b-68ba81f88402 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:37.940890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3692-19f9-40b1-b98b-68ba81f88402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:37.940890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3692-19f9-40b1-b98b-68ba81f88402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:37.941642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab3692-19f9-40b1-b98b-68ba81f88402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:37.943210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab3692-19f9-40b1-b98b-68ba81f88402 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:37.954132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1267/2526] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:28:37.958327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189665ef-8b25-43c5-bde0-16233c6ab5ca admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:37.969200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189665ef-8b25-43c5-bde0-16233c6ab5ca admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:37.974186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525c446-1d9f-4dc0-aa6d-a86d5744ef74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:37.975533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525c446-1d9f-4dc0-aa6d-a86d5744ef74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:37.975984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525c446-1d9f-4dc0-aa6d-a86d5744ef74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:37.976355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525c446-1d9f-4dc0-aa6d-a86d5744ef74 admin admin] PATH_INFO: `/v3/projects/d3a7edc4db5c44f68377d44e9f20d6bb/users/eea90c1514254bf68bbde782cebbb32e/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:37.978530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525c446-1d9f-4dc0-aa6d-a86d5744ef74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a7edc4db5c44f68377d44e9f20d6bb, user_id=eea90c1514254bf68bbde782cebbb32e, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:37.981382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525c446-1d9f-4dc0-aa6d-a86d5744ef74 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:37.997840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1260/2527] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:28:37 2022] PUT /identity/v3/projects/d3a7edc4db5c44f68377d44e9f20d6bb/users/eea90c1514254bf68bbde782cebbb32e/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:28:38.004095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8c38b-7d0a-48b5-8118-2241cdc05424 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:38.004727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8c38b-7d0a-48b5-8118-2241cdc05424 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:38.005199 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8c38b-7d0a-48b5-8118-2241cdc05424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:38.041619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fba8c38b-7d0a-48b5-8118-2241cdc05424 None None] MFA Rules not processed for user `eea90c1514254bf68bbde782cebbb32e`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:38.043085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fba8c38b-7d0a-48b5-8118-2241cdc05424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:38.090153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:38.138631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1268/2528] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:28:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:38.150073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525c446-1d9f-4dc0-aa6d-a86d5744ef74 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:38.151976 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a525c446-1d9f-4dc0-aa6d-a86d5744ef74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:38.186462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:38.267190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525c446-1d9f-4dc0-aa6d-a86d5744ef74 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:38.272190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00b514-50b6-435f-9e0b-649f37f4fc84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:38.272980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00b514-50b6-435f-9e0b-649f37f4fc84 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:38.273405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00b514-50b6-435f-9e0b-649f37f4fc84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:38.274033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00b514-50b6-435f-9e0b-649f37f4fc84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:38.337876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00b514-50b6-435f-9e0b-649f37f4fc84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:38.339433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00b514-50b6-435f-9e0b-649f37f4fc84 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:38.348982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1261/2529] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:44.704831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8c38b-7d0a-48b5-8118-2241cdc05424 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:44.706215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fba8c38b-7d0a-48b5-8118-2241cdc05424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:44.718647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8c38b-7d0a-48b5-8118-2241cdc05424 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:44.722353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e3ec3-0688-49e8-a567-3af6601e726a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:44.722960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e3ec3-0688-49e8-a567-3af6601e726a service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:44.723240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e3ec3-0688-49e8-a567-3af6601e726a service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:44.723515 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e3ec3-0688-49e8-a567-3af6601e726a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:44.733106 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e3ec3-0688-49e8-a567-3af6601e726a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:44.734776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e3ec3-0688-49e8-a567-3af6601e726a service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:44.744731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1269/2530] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:28:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:28:48.344484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=43 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:54.740400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=18 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:28:57.429330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa44ee7-0cfd-461a-a1a6-6648b07c4ed1 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:57.429794 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa44ee7-0cfd-461a-a1a6-6648b07c4ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:57.429912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa44ee7-0cfd-461a-a1a6-6648b07c4ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:57.449192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-bfa44ee7-0cfd-461a-a1a6-6648b07c4ed1 None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:28:57.450358 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bfa44ee7-0cfd-461a-a1a6-6648b07c4ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:57.503887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1262/2531] 10.222.0.25 () {58 vars in 967 bytes} [Sat Aug 6 01:28:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:28:57.516746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e3ec3-0688-49e8-a567-3af6601e726a service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:28:57.518122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-286e3ec3-0688-49e8-a567-3af6601e726a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:28:57.591538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e3ec3-0688-49e8-a567-3af6601e726a service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:28:57.596543 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee48f7c-256e-4252-8d51-3f324b61c521 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:28:57.597152 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee48f7c-256e-4252-8d51-3f324b61c521 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:28:57.597455 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee48f7c-256e-4252-8d51-3f324b61c521 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:28:57.597726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee48f7c-256e-4252-8d51-3f324b61c521 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:28:57.606230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee48f7c-256e-4252-8d51-3f324b61c521 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:28:57.607825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee48f7c-256e-4252-8d51-3f324b61c521 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:28:57.616529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1270/2532] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:28:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:29:07.497332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=25 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:07.612842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=32 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:07.795318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa44ee7-0cfd-461a-a1a6-6648b07c4ed1 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:07.805563 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa44ee7-0cfd-461a-a1a6-6648b07c4ed1 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:07.809223 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83c397-6ff6-452b-82e0-67af312c3dd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:07.811560 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83c397-6ff6-452b-82e0-67af312c3dd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:07.811560 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83c397-6ff6-452b-82e0-67af312c3dd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:07.811560 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83c397-6ff6-452b-82e0-67af312c3dd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:07.812515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1f83c397-6ff6-452b-82e0-67af312c3dd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:07.870740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83c397-6ff6-452b-82e0-67af312c3dd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:07.872231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83c397-6ff6-452b-82e0-67af312c3dd8 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:07.881209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1263/2533] 10.222.0.25 () {60 vars in 1396 bytes} [Sat Aug 6 01:29:07 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:12.480318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee48f7c-256e-4252-8d51-3f324b61c521 service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:12.543951 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee48f7c-256e-4252-8d51-3f324b61c521 service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:12.548459 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:12.549089 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:12.549365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:12.549635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] PATH_INFO: `/v3/users/a159c9b1203544c8942c0e7d15eafd3d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:12.550771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a159c9b1203544c8942c0e7d15eafd3d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:12.552703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:12.718286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a159c9b1203544c8942c0e7d15eafd3d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:12.725039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a159c9b1203544c8942c0e7d15eafd3d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:12.789263 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a159c9b1203544c8942c0e7d15eafd3d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:12.789877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a159c9b1203544c8942c0e7d15eafd3d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:12.802759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a159c9b1203544c8942c0e7d15eafd3d'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:12.808678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1271/2534] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:29:12 2022] DELETE /identity/v3/users/a159c9b1203544c8942c0e7d15eafd3d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:12.813140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0cde3-e9b4-4bd6-ba93-e7df0fd23bbd None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:12.813598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0cde3-e9b4-4bd6-ba93-e7df0fd23bbd None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:12.813696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0cde3-e9b4-4bd6-ba93-e7df0fd23bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:12.832743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-26e0cde3-e9b4-4bd6-ba93-e7df0fd23bbd None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:12.833943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-26e0cde3-e9b4-4bd6-ba93-e7df0fd23bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:12.891264 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1264/2535] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:12.902404 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:12.903831 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:12.962093 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6b30f-f00d-4980-be5b-7c92cdc19d3b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:12.966327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a64637-8299-4656-87d6-da44583288e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:12.966941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a64637-8299-4656-87d6-da44583288e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:12.967222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a64637-8299-4656-87d6-da44583288e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:12.967501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a64637-8299-4656-87d6-da44583288e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:12.976401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a64637-8299-4656-87d6-da44583288e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:12.977770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a64637-8299-4656-87d6-da44583288e2 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:12.987362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1272/2536] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:13.402764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0cde3-e9b4-4bd6-ba93-e7df0fd23bbd None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:13.404323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-26e0cde3-e9b4-4bd6-ba93-e7df0fd23bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:13.416142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0cde3-e9b4-4bd6-ba93-e7df0fd23bbd None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:13.420989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:13.421643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:13.421920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:13.422202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] PATH_INFO: `/v3/projects/19bad7d46cf84b83bfd77b063744298d` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:13.423362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] RBAC: Authorizing `identity:delete_project(project_id=19bad7d46cf84b83bfd77b063744298d)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:13.426278 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:13.531933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19bad7d46cf84b83bfd77b063744298d', 'request_id': 'req-512ecddd-2c19-4f26-bc53-54421095499a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:13.539166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19bad7d46cf84b83bfd77b063744298d', 'request_id': 'req-512ecddd-2c19-4f26-bc53-54421095499a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:13.595187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19bad7d46cf84b83bfd77b063744298d', 'request_id': 'req-512ecddd-2c19-4f26-bc53-54421095499a'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:13.606803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] The token cache is being invalidate because project 19bad7d46cf84b83bfd77b063744298d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:29:13.607093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:13.607713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:13.609503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1265/2537] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:29:13 2022] DELETE /identity/v3/projects/19bad7d46cf84b83bfd77b063744298d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:13.613441 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a64637-8299-4656-87d6-da44583288e2 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:13.623489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a64637-8299-4656-87d6-da44583288e2 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:13.641936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:13.679375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:13.680080 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:13.680361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:13.680631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] PATH_INFO: `/v3/users/eea90c1514254bf68bbde782cebbb32e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:13.682384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea90c1514254bf68bbde782cebbb32e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:13.684427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:13.834585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea90c1514254bf68bbde782cebbb32e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:13.841063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea90c1514254bf68bbde782cebbb32e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:13.907490 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea90c1514254bf68bbde782cebbb32e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:13.908173 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea90c1514254bf68bbde782cebbb32e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:13.920421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-dc6cfe27-e61d-4b25-a970-09be034b54a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea90c1514254bf68bbde782cebbb32e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:13.926153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1273/2538] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:29:13 2022] DELETE /identity/v3/users/eea90c1514254bf68bbde782cebbb32e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:14.316902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:14.318438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:14.332327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ecddd-2c19-4f26-bc53-54421095499a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:14.355036 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:14.392454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:14.393103 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:14.393387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:14.393663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] PATH_INFO: `/v3/projects/d3a7edc4db5c44f68377d44e9f20d6bb` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:14.394833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a7edc4db5c44f68377d44e9f20d6bb)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:14.396798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:14.508384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a7edc4db5c44f68377d44e9f20d6bb', 'request_id': 'req-d93492c0-ffae-4187-9cee-c915171b591c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:14.514420 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a7edc4db5c44f68377d44e9f20d6bb', 'request_id': 'req-d93492c0-ffae-4187-9cee-c915171b591c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:14.604397 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a7edc4db5c44f68377d44e9f20d6bb', 'request_id': 'req-d93492c0-ffae-4187-9cee-c915171b591c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:14.612942 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] The token cache is being invalidate because project d3a7edc4db5c44f68377d44e9f20d6bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:29:14.613256 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:14.613899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d93492c0-ffae-4187-9cee-c915171b591c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:14.615457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1266/2539] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:29:14 2022] DELETE /identity/v3/projects/d3a7edc4db5c44f68377d44e9f20d6bb => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:29:14.620516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e8df8-f317-4aee-9145-4cad1ea6f4b0 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:14.620986 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e8df8-f317-4aee-9145-4cad1ea6f4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:14.621265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e8df8-f317-4aee-9145-4cad1ea6f4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:14.641797 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-de3e8df8-f317-4aee-9145-4cad1ea6f4b0 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:14.642999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-de3e8df8-f317-4aee-9145-4cad1ea6f4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:14.700359 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1274/2540] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:14.708220 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f25ba2-c80f-4406-9ee0-e22db3277fcf None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:14.708510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f25ba2-c80f-4406-9ee0-e22db3277fcf None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:14.708790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f25ba2-c80f-4406-9ee0-e22db3277fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:14.728430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b1f25ba2-c80f-4406-9ee0-e22db3277fcf None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:14.729646 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b1f25ba2-c80f-4406-9ee0-e22db3277fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:14.737586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1267/2541] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:14.741807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e8df8-f317-4aee-9145-4cad1ea6f4b0 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:14.751458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e8df8-f317-4aee-9145-4cad1ea6f4b0 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:14.756029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea46da-aa29-427d-974c-61116b7f83d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:14.756634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea46da-aa29-427d-974c-61116b7f83d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:14.756919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea46da-aa29-427d-974c-61116b7f83d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:14.757196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea46da-aa29-427d-974c-61116b7f83d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:14.757890 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea46da-aa29-427d-974c-61116b7f83d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:29:14.758203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea46da-aa29-427d-974c-61116b7f83d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:14.760311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea46da-aa29-427d-974c-61116b7f83d5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:14.771545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1275/2542] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:29:14.776357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f25ba2-c80f-4406-9ee0-e22db3277fcf None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:14.785427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f25ba2-c80f-4406-9ee0-e22db3277fcf None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:14.789987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac37f7-87fd-4154-914c-aaa554c2c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:14.790701 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac37f7-87fd-4154-914c-aaa554c2c7ff admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:14.790983 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac37f7-87fd-4154-914c-aaa554c2c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:14.791257 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac37f7-87fd-4154-914c-aaa554c2c7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:14.791965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac37f7-87fd-4154-914c-aaa554c2c7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:14.793862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac37f7-87fd-4154-914c-aaa554c2c7ff admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:14.891852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1268/2543] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:29:14 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:29:14.896210 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea46da-aa29-427d-974c-61116b7f83d5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:14.910383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea46da-aa29-427d-974c-61116b7f83d5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:14.935787 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:14.965888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93304367-fb94-48d3-9a6d-9bd9f0f93473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:14.966595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93304367-fb94-48d3-9a6d-9bd9f0f93473 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:14.966932 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93304367-fb94-48d3-9a6d-9bd9f0f93473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:14.967211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93304367-fb94-48d3-9a6d-9bd9f0f93473 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:14.967903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93304367-fb94-48d3-9a6d-9bd9f0f93473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:14.969808 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93304367-fb94-48d3-9a6d-9bd9f0f93473 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:15.033090 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1276/2544] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:29:14 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:15.037416 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac37f7-87fd-4154-914c-aaa554c2c7ff admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:15.048021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac37f7-87fd-4154-914c-aaa554c2c7ff admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:15.054122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bcc31-cac0-4832-ac56-40b3e16e5661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:15.055681 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcc31-cac0-4832-ac56-40b3e16e5661 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:15.056865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcc31-cac0-4832-ac56-40b3e16e5661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:15.057142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcc31-cac0-4832-ac56-40b3e16e5661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:15.057835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bcc31-cac0-4832-ac56-40b3e16e5661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:15.059440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bcc31-cac0-4832-ac56-40b3e16e5661 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:15.072362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1269/2545] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:29:15.075494 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93304367-fb94-48d3-9a6d-9bd9f0f93473 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:15.087215 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93304367-fb94-48d3-9a6d-9bd9f0f93473 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:15.092258 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabff933-6d62-4a93-b31f-5354c0d69070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:15.092881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabff933-6d62-4a93-b31f-5354c0d69070 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:15.093164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabff933-6d62-4a93-b31f-5354c0d69070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:15.093436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabff933-6d62-4a93-b31f-5354c0d69070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:15.094154 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabff933-6d62-4a93-b31f-5354c0d69070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:15.095851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabff933-6d62-4a93-b31f-5354c0d69070 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:15.107111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1277/2546] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:29:15.111531 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bcc31-cac0-4832-ac56-40b3e16e5661 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:15.122404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bcc31-cac0-4832-ac56-40b3e16e5661 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:15.127016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe95143-e0f5-4c3e-bf96-1a711d34b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:15.127667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe95143-e0f5-4c3e-bf96-1a711d34b3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:15.128064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe95143-e0f5-4c3e-bf96-1a711d34b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:15.128355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe95143-e0f5-4c3e-bf96-1a711d34b3d8 admin admin] PATH_INFO: `/v3/projects/a68f030c5c624770ae573dc74f3ee38b/users/8b0f9f52470b48d0b39c53ce01de53bd/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:15.144425 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe95143-e0f5-4c3e-bf96-1a711d34b3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68f030c5c624770ae573dc74f3ee38b, user_id=8b0f9f52470b48d0b39c53ce01de53bd, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:15.147032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe95143-e0f5-4c3e-bf96-1a711d34b3d8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:15.217633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1270/2547] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:29:15 2022] PUT /identity/v3/projects/a68f030c5c624770ae573dc74f3ee38b/users/8b0f9f52470b48d0b39c53ce01de53bd/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:15.229506 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1f945-9945-4d21-b5ce-b9613a17718f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:15.229800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1f945-9945-4d21-b5ce-b9613a17718f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:15.230085 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1f945-9945-4d21-b5ce-b9613a17718f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:15.258657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-eef1f945-9945-4d21-b5ce-b9613a17718f None None] MFA Rules not processed for user `8b0f9f52470b48d0b39c53ce01de53bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:15.259953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-eef1f945-9945-4d21-b5ce-b9613a17718f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:15.322567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1278/2548] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:15.333659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe95143-e0f5-4c3e-bf96-1a711d34b3d8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:15.335369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9fe95143-e0f5-4c3e-bf96-1a711d34b3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:15.390474 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:15.394187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe95143-e0f5-4c3e-bf96-1a711d34b3d8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:15.403105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d0be4-aa5b-4664-b321-c6506831482c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:15.403105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d0be4-aa5b-4664-b321-c6506831482c service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:15.403105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d0be4-aa5b-4664-b321-c6506831482c service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:15.403105 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d0be4-aa5b-4664-b321-c6506831482c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:15.407317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d0be4-aa5b-4664-b321-c6506831482c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:15.408770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d0be4-aa5b-4664-b321-c6506831482c service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:15.417887 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1271/2549] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:15.437221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1f945-9945-4d21-b5ce-b9613a17718f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:15.509611 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1f945-9945-4d21-b5ce-b9613a17718f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:15.514327 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:15.514998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:15.515281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:15.515557 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] PATH_INFO: `/v3/users/8b0f9f52470b48d0b39c53ce01de53bd` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:15.516841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0f9f52470b48d0b39c53ce01de53bd)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:15.518817 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:15.700240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:15.712801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0f9f52470b48d0b39c53ce01de53bd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:15.716091 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0f9f52470b48d0b39c53ce01de53bd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:15.927513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0f9f52470b48d0b39c53ce01de53bd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:15.928102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0f9f52470b48d0b39c53ce01de53bd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:15.940248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0f9f52470b48d0b39c53ce01de53bd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:15.946212 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1279/2550] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:29:15 2022] DELETE /identity/v3/users/8b0f9f52470b48d0b39c53ce01de53bd => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:15.951198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e846-c9df-431b-a2d1-c66b31587471 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:15.951489 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e846-c9df-431b-a2d1-c66b31587471 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:15.951810 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e846-c9df-431b-a2d1-c66b31587471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:15.972316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-fde8e846-c9df-431b-a2d1-c66b31587471 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:15.973543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fde8e846-c9df-431b-a2d1-c66b31587471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:16.031209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1272/2551] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:16.040991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:16.044683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:16.099799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f7c22-fb5b-41b9-ac29-70f6af0c56a3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:16.103423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e64d7b-b0d7-43da-a649-a9ca349c4680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:16.104075 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e64d7b-b0d7-43da-a649-a9ca349c4680 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:16.104357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e64d7b-b0d7-43da-a649-a9ca349c4680 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:16.104628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e64d7b-b0d7-43da-a649-a9ca349c4680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:16.113813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e64d7b-b0d7-43da-a649-a9ca349c4680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:16.115218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e64d7b-b0d7-43da-a649-a9ca349c4680 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:16.124646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1280/2552] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:16.515066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8e846-c9df-431b-a2d1-c66b31587471 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:16.518021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-fde8e846-c9df-431b-a2d1-c66b31587471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:16.528689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8e846-c9df-431b-a2d1-c66b31587471 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:16.533307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:16.533952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:16.534232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:16.534507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] PATH_INFO: `/v3/projects/a68f030c5c624770ae573dc74f3ee38b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:16.535655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68f030c5c624770ae573dc74f3ee38b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:16.537602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:16.685792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68f030c5c624770ae573dc74f3ee38b', 'request_id': 'req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:16.691251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68f030c5c624770ae573dc74f3ee38b', 'request_id': 'req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:16.749627 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68f030c5c624770ae573dc74f3ee38b', 'request_id': 'req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:16.761206 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] The token cache is being invalidate because project a68f030c5c624770ae573dc74f3ee38b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:29:16.761438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:16.762073 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d7cfe955-978a-46f3-8fbc-1b5af91e81a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:16.763562 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1273/2553] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:29:16 2022] DELETE /identity/v3/projects/a68f030c5c624770ae573dc74f3ee38b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:16.770207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fc5d3-10ff-41ee-ae55-a1ad95668a93 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:16.770207 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fc5d3-10ff-41ee-ae55-a1ad95668a93 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:16.770677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fc5d3-10ff-41ee-ae55-a1ad95668a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:16.789674 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-585fc5d3-10ff-41ee-ae55-a1ad95668a93 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:16.793013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-585fc5d3-10ff-41ee-ae55-a1ad95668a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:16.857697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1281/2554] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:16.865697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e81cd-e7df-4ab2-b8b5-bf45b34aa079 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:16.866019 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e81cd-e7df-4ab2-b8b5-bf45b34aa079 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:16.866299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e81cd-e7df-4ab2-b8b5-bf45b34aa079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:16.886896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-138e81cd-e7df-4ab2-b8b5-bf45b34aa079 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:16.888142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-138e81cd-e7df-4ab2-b8b5-bf45b34aa079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:16.896069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1274/2555] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:16.900428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fc5d3-10ff-41ee-ae55-a1ad95668a93 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:16.909985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fc5d3-10ff-41ee-ae55-a1ad95668a93 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:16.914565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601975e-8e01-4dce-a266-e12dc9574abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:16.915692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601975e-8e01-4dce-a266-e12dc9574abf admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:16.916166 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601975e-8e01-4dce-a266-e12dc9574abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:16.916353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601975e-8e01-4dce-a266-e12dc9574abf admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:16.917050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601975e-8e01-4dce-a266-e12dc9574abf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:29:16.917357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601975e-8e01-4dce-a266-e12dc9574abf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:16.918943 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601975e-8e01-4dce-a266-e12dc9574abf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:16.930226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1282/2556] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:29:16.935195 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e81cd-e7df-4ab2-b8b5-bf45b34aa079 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:16.944386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e81cd-e7df-4ab2-b8b5-bf45b34aa079 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:16.948968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8ffd2-2a96-4496-8828-b2e1de93b2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:16.949559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8ffd2-2a96-4496-8828-b2e1de93b2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:16.949866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8ffd2-2a96-4496-8828-b2e1de93b2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:16.950133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8ffd2-2a96-4496-8828-b2e1de93b2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:16.950799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8ffd2-2a96-4496-8828-b2e1de93b2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:16.952715 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8ffd2-2a96-4496-8828-b2e1de93b2e1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:17.065707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1275/2557] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:29:16 2022] POST /identity/v3/projects => generated 359 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:17.070188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601975e-8e01-4dce-a266-e12dc9574abf admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:17.082304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601975e-8e01-4dce-a266-e12dc9574abf admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:17.130071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:17.144100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c219e-da46-4936-bc3b-b3cba9b3c47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:17.144100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c219e-da46-4936-bc3b-b3cba9b3c47d admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:17.144331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c219e-da46-4936-bc3b-b3cba9b3c47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:17.144461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c219e-da46-4936-bc3b-b3cba9b3c47d admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:17.145127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c219e-da46-4936-bc3b-b3cba9b3c47d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:17.147052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c219e-da46-4936-bc3b-b3cba9b3c47d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:17.247065 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1283/2558] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:29:17 2022] POST /identity/v3/users => generated 280 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:17.251630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8ffd2-2a96-4496-8828-b2e1de93b2e1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:17.262181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8ffd2-2a96-4496-8828-b2e1de93b2e1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:17.266724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbff1e-626a-4689-9d8b-94c7f31b547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:17.267328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbff1e-626a-4689-9d8b-94c7f31b547b admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:17.267607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbff1e-626a-4689-9d8b-94c7f31b547b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:17.267922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cbff1e-626a-4689-9d8b-94c7f31b547b admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:17.268709 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbff1e-626a-4689-9d8b-94c7f31b547b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:17.270347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cbff1e-626a-4689-9d8b-94c7f31b547b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:17.280713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1276/2559] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:29:17.284709 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c219e-da46-4936-bc3b-b3cba9b3c47d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:17.295620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c219e-da46-4936-bc3b-b3cba9b3c47d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:17.300519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c3f4f-6589-4e75-aed6-b34bbb27aacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:17.301191 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c3f4f-6589-4e75-aed6-b34bbb27aacf admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:17.301478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c3f4f-6589-4e75-aed6-b34bbb27aacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:17.301757 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c3f4f-6589-4e75-aed6-b34bbb27aacf admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:17.302492 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c3f4f-6589-4e75-aed6-b34bbb27aacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:17.304132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c3f4f-6589-4e75-aed6-b34bbb27aacf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:17.314901 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1284/2560] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:29:17.319196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbff1e-626a-4689-9d8b-94c7f31b547b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:17.329078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cbff1e-626a-4689-9d8b-94c7f31b547b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:17.333685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be09c57-ca66-427a-9274-3b3dbd3ebddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:17.334337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be09c57-ca66-427a-9274-3b3dbd3ebddd admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:17.334620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be09c57-ca66-427a-9274-3b3dbd3ebddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:17.334898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be09c57-ca66-427a-9274-3b3dbd3ebddd admin admin] PATH_INFO: `/v3/projects/87559865b7e84e66886707c87a77822a/users/19fe109f317e46a8b7ebc2ef67499b4a/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:17.354276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be09c57-ca66-427a-9274-3b3dbd3ebddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=87559865b7e84e66886707c87a77822a, user_id=19fe109f317e46a8b7ebc2ef67499b4a, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:17.358802 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be09c57-ca66-427a-9274-3b3dbd3ebddd admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:17.551509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1277/2561] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:29:17 2022] PUT /identity/v3/projects/87559865b7e84e66886707c87a77822a/users/19fe109f317e46a8b7ebc2ef67499b4a/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:17.569632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664e788-6089-4b95-8455-38d297633125 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:17.569920 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664e788-6089-4b95-8455-38d297633125 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:17.570188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664e788-6089-4b95-8455-38d297633125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:17.602168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f664e788-6089-4b95-8455-38d297633125 None None] MFA Rules not processed for user `19fe109f317e46a8b7ebc2ef67499b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:17.603491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f664e788-6089-4b95-8455-38d297633125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:17.667303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1285/2562] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:17.702553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be09c57-ca66-427a-9274-3b3dbd3ebddd admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:17.704267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1be09c57-ca66-427a-9274-3b3dbd3ebddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:17.718743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:17.761499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be09c57-ca66-427a-9274-3b3dbd3ebddd admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:17.765133 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c357d-2dc0-4c53-87ee-f0f27fbbf97d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:17.765811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c357d-2dc0-4c53-87ee-f0f27fbbf97d service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:17.766097 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c357d-2dc0-4c53-87ee-f0f27fbbf97d service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:17.766373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c357d-2dc0-4c53-87ee-f0f27fbbf97d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:17.775440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c357d-2dc0-4c53-87ee-f0f27fbbf97d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:17.776907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c357d-2dc0-4c53-87ee-f0f27fbbf97d service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:17.785731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1278/2563] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:17 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:17.801019 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664e788-6089-4b95-8455-38d297633125 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:17.861844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664e788-6089-4b95-8455-38d297633125 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:17.866333 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:17.866960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:17.867239 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:17.867513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] PATH_INFO: `/v3/users/19fe109f317e46a8b7ebc2ef67499b4a` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:17.869223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19fe109f317e46a8b7ebc2ef67499b4a)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:17.871115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:18.064299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19fe109f317e46a8b7ebc2ef67499b4a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:18.072119 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19fe109f317e46a8b7ebc2ef67499b4a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:18.073823 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:18.136108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19fe109f317e46a8b7ebc2ef67499b4a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:18.136760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19fe109f317e46a8b7ebc2ef67499b4a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:18.149144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19fe109f317e46a8b7ebc2ef67499b4a'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:18.155017 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1286/2564] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:29:17 2022] DELETE /identity/v3/users/19fe109f317e46a8b7ebc2ef67499b4a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:18.159559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276efaf6-ed08-46f5-b3a1-475e11e7cc32 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:18.159893 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276efaf6-ed08-46f5-b3a1-475e11e7cc32 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:18.160178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276efaf6-ed08-46f5-b3a1-475e11e7cc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:18.178963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-276efaf6-ed08-46f5-b3a1-475e11e7cc32 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:18.180204 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-276efaf6-ed08-46f5-b3a1-475e11e7cc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:18.239624 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1279/2565] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:18.249696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:18.251103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:18.307810 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82548eaa-d733-4a81-b581-77bc15f12a27 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:18.311826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410929f8-b421-4240-b23e-dce22cdf4e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:18.312446 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410929f8-b421-4240-b23e-dce22cdf4e26 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:18.312725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410929f8-b421-4240-b23e-dce22cdf4e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:18.312999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410929f8-b421-4240-b23e-dce22cdf4e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:18.325978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410929f8-b421-4240-b23e-dce22cdf4e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:18.325978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410929f8-b421-4240-b23e-dce22cdf4e26 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:18.332902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1287/2566] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:18.789172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276efaf6-ed08-46f5-b3a1-475e11e7cc32 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:18.791203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-276efaf6-ed08-46f5-b3a1-475e11e7cc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:18.803294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276efaf6-ed08-46f5-b3a1-475e11e7cc32 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:18.808499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:18.809151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:18.809436 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:18.809710 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] PATH_INFO: `/v3/projects/87559865b7e84e66886707c87a77822a` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:18.810888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87559865b7e84e66886707c87a77822a)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:18.812845 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:18.918832 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87559865b7e84e66886707c87a77822a', 'request_id': 'req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:18.925657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87559865b7e84e66886707c87a77822a', 'request_id': 'req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:19.006867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87559865b7e84e66886707c87a77822a', 'request_id': 'req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:19.018801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] The token cache is being invalidate because project 87559865b7e84e66886707c87a77822a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:29:19.019090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:19.019716 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-a64f9bd2-3ad6-4068-93f7-47a32d2dab82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:19.021242 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1280/2567] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:29:18 2022] DELETE /identity/v3/projects/87559865b7e84e66886707c87a77822a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:19.026663 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b734c-31ab-4b30-b74f-59e1eaa418a0 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.027137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b734c-31ab-4b30-b74f-59e1eaa418a0 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.027250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b734c-31ab-4b30-b74f-59e1eaa418a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.047319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-145b734c-31ab-4b30-b74f-59e1eaa418a0 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:19.048531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-145b734c-31ab-4b30-b74f-59e1eaa418a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:19.232914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1288/2568] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:29:19.240621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1473bd5-1477-4700-92db-203b947ec51f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.240911 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1473bd5-1477-4700-92db-203b947ec51f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.241188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1473bd5-1477-4700-92db-203b947ec51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.261754 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-a1473bd5-1477-4700-92db-203b947ec51f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:19.262962 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-a1473bd5-1477-4700-92db-203b947ec51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:19.270672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1281/2569] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:19.275476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b734c-31ab-4b30-b74f-59e1eaa418a0 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.284927 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b734c-31ab-4b30-b74f-59e1eaa418a0 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.289419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2217f-4bff-4f94-be3e-d77989f10c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.290012 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2217f-4bff-4f94-be3e-d77989f10c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.290352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2217f-4bff-4f94-be3e-d77989f10c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.290625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2217f-4bff-4f94-be3e-d77989f10c89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.291312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2217f-4bff-4f94-be3e-d77989f10c89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:29:19.291612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2217f-4bff-4f94-be3e-d77989f10c89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.293506 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2217f-4bff-4f94-be3e-d77989f10c89 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.304965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1289/2570] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:29:19.309703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1473bd5-1477-4700-92db-203b947ec51f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.327302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1473bd5-1477-4700-92db-203b947ec51f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.327302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd873ba-73db-4c0b-a01d-7d69c739d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.327302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd873ba-73db-4c0b-a01d-7d69c739d14f admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.327302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd873ba-73db-4c0b-a01d-7d69c739d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.327965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd873ba-73db-4c0b-a01d-7d69c739d14f admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.327965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd873ba-73db-4c0b-a01d-7d69c739d14f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.328226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd873ba-73db-4c0b-a01d-7d69c739d14f admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.552188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1282/2571] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:29:19 2022] POST /identity/v3/projects => generated 365 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:19.557963 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2217f-4bff-4f94-be3e-d77989f10c89 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.568953 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2217f-4bff-4f94-be3e-d77989f10c89 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.637304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7ab4c-7a83-4434-b7fe-8a3e5875db23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.638600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7ab4c-7a83-4434-b7fe-8a3e5875db23 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.638600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7ab4c-7a83-4434-b7fe-8a3e5875db23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.638600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7ab4c-7a83-4434-b7fe-8a3e5875db23 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.639111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7ab4c-7a83-4434-b7fe-8a3e5875db23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.641046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7ab4c-7a83-4434-b7fe-8a3e5875db23 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.650841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:19.766900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1290/2572] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:29:19 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:19.770997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd873ba-73db-4c0b-a01d-7d69c739d14f admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.781119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd873ba-73db-4c0b-a01d-7d69c739d14f admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.785700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa80759-7302-4c51-88f6-1244027f08a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.786341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa80759-7302-4c51-88f6-1244027f08a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.786622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa80759-7302-4c51-88f6-1244027f08a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.786897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa80759-7302-4c51-88f6-1244027f08a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.787612 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa80759-7302-4c51-88f6-1244027f08a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.789293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa80759-7302-4c51-88f6-1244027f08a9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.791866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:19.800255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1283/2573] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:29:19.804150 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7ab4c-7a83-4434-b7fe-8a3e5875db23 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.814785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7ab4c-7a83-4434-b7fe-8a3e5875db23 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.819924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700dcec-7519-4956-8c82-10ef901fc505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.820522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700dcec-7519-4956-8c82-10ef901fc505 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.823733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700dcec-7519-4956-8c82-10ef901fc505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.823733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700dcec-7519-4956-8c82-10ef901fc505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.823733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700dcec-7519-4956-8c82-10ef901fc505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.823733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700dcec-7519-4956-8c82-10ef901fc505 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.833365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1291/2574] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:29:19.837375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa80759-7302-4c51-88f6-1244027f08a9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:19.847736 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa80759-7302-4c51-88f6-1244027f08a9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:19.852315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf7ffc-8443-4ed0-87ca-abf4dd5cc657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:19.853521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf7ffc-8443-4ed0-87ca-abf4dd5cc657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.853804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf7ffc-8443-4ed0-87ca-abf4dd5cc657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.854181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf7ffc-8443-4ed0-87ca-abf4dd5cc657 admin admin] PATH_INFO: `/v3/projects/bd630b3de83e462e82ccdc98d2413842/users/07ff90c4e586488eaa13365fbf7cd339/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.869057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf7ffc-8443-4ed0-87ca-abf4dd5cc657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd630b3de83e462e82ccdc98d2413842, user_id=07ff90c4e586488eaa13365fbf7cd339, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:19.871746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf7ffc-8443-4ed0-87ca-abf4dd5cc657 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:19.926004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1284/2575] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:29:19 2022] PUT /identity/v3/projects/bd630b3de83e462e82ccdc98d2413842/users/07ff90c4e586488eaa13365fbf7cd339/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:19.937350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35836db8-0d80-4dc0-b018-0b5991e3f25d None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:19.937910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35836db8-0d80-4dc0-b018-0b5991e3f25d None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:19.944734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35836db8-0d80-4dc0-b018-0b5991e3f25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:19.965144 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-35836db8-0d80-4dc0-b018-0b5991e3f25d None None] MFA Rules not processed for user `07ff90c4e586488eaa13365fbf7cd339`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:19.966355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-35836db8-0d80-4dc0-b018-0b5991e3f25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:20.032529 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1292/2576] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:20.043984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf7ffc-8443-4ed0-87ca-abf4dd5cc657 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:20.045773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4baf7ffc-8443-4ed0-87ca-abf4dd5cc657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:20.103850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf7ffc-8443-4ed0-87ca-abf4dd5cc657 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:20.107465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed5021-4fe6-4d27-8a70-5b8f2defea9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:20.108123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5021-4fe6-4d27-8a70-5b8f2defea9d service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:20.108409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5021-4fe6-4d27-8a70-5b8f2defea9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:20.108950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5021-4fe6-4d27-8a70-5b8f2defea9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:20.117343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed5021-4fe6-4d27-8a70-5b8f2defea9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:20.119283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed5021-4fe6-4d27-8a70-5b8f2defea9d service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:20.128251 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1285/2577] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:30.029509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:INSERT=1,keystone:SELECT=39 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:30.124119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:34.341079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35836db8-0d80-4dc0-b018-0b5991e3f25d None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:34.342481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-35836db8-0d80-4dc0-b018-0b5991e3f25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:34.398354 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35836db8-0d80-4dc0-b018-0b5991e3f25d None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:34.402572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aea2b9-4017-4290-a886-6ed5416413aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:34.403223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aea2b9-4017-4290-a886-6ed5416413aa service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:34.403486 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aea2b9-4017-4290-a886-6ed5416413aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:34.403758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aea2b9-4017-4290-a886-6ed5416413aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:34.405550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-07aea2b9-4017-4290-a886-6ed5416413aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:34.466835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aea2b9-4017-4290-a886-6ed5416413aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:34.468314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aea2b9-4017-4290-a886-6ed5416413aa service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:34.477699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1293/2578] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:38.310723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed5021-4fe6-4d27-8a70-5b8f2defea9d service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:38.312374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6fed5021-4fe6-4d27-8a70-5b8f2defea9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:38.327588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed5021-4fe6-4d27-8a70-5b8f2defea9d service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:38.332152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:38.335669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:38.335669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:38.335669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] PATH_INFO: `/v3/users/b07c458f1a5c4391aaaa79af49a236c7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:38.335669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07c458f1a5c4391aaaa79af49a236c7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:38.336418 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:38.531763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07c458f1a5c4391aaaa79af49a236c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:38.538584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07c458f1a5c4391aaaa79af49a236c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:38.592550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07c458f1a5c4391aaaa79af49a236c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:38.593168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07c458f1a5c4391aaaa79af49a236c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:38.593772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1b7ddff7-4fa6-4d4f-8c14-9f3aaa3cca50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07c458f1a5c4391aaaa79af49a236c7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:38.613130 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1286/2579] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:29:38 2022] DELETE /identity/v3/users/b07c458f1a5c4391aaaa79af49a236c7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:38.865267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aea2b9-4017-4290-a886-6ed5416413aa service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:38.877461 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aea2b9-4017-4290-a886-6ed5416413aa service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:38.941286 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:38.941967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:38.942245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:38.942522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] PATH_INFO: `/v3/projects/7e49a8a993484e33b0892a6a86d3d196` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:38.943678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e49a8a993484e33b0892a6a86d3d196)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:38.945604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:39.030153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e49a8a993484e33b0892a6a86d3d196', 'request_id': 'req-8161903e-a166-4ec2-a3e5-ca9660cd137c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:39.032035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:39.036656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e49a8a993484e33b0892a6a86d3d196', 'request_id': 'req-8161903e-a166-4ec2-a3e5-ca9660cd137c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:39.042118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e49a8a993484e33b0892a6a86d3d196', 'request_id': 'req-8161903e-a166-4ec2-a3e5-ca9660cd137c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:39.097667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] The token cache is being invalidate because project 7e49a8a993484e33b0892a6a86d3d196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:29:39.097957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:39.098572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-8161903e-a166-4ec2-a3e5-ca9660cd137c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:39.100113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1294/2580] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:29:38 2022] DELETE /identity/v3/projects/7e49a8a993484e33b0892a6a86d3d196 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:29:39.105259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d51fcc-ff45-4887-86b0-c8e3831eff4f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:39.105554 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d51fcc-ff45-4887-86b0-c8e3831eff4f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:39.105856 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d51fcc-ff45-4887-86b0-c8e3831eff4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:39.125433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-24d51fcc-ff45-4887-86b0-c8e3831eff4f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:39.126650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-24d51fcc-ff45-4887-86b0-c8e3831eff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:39.188592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1287/2581] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:39.198654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1681d-3eeb-4a5a-b280-13e21666412c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:39.199516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1681d-3eeb-4a5a-b280-13e21666412c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:39.199829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1681d-3eeb-4a5a-b280-13e21666412c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:39.219723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1ad1681d-3eeb-4a5a-b280-13e21666412c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:39.221034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1ad1681d-3eeb-4a5a-b280-13e21666412c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:39.228826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1295/2582] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:39.233093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d51fcc-ff45-4887-86b0-c8e3831eff4f None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:39.243186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d51fcc-ff45-4887-86b0-c8e3831eff4f None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:39.247870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8681f45-79c3-4a1e-9068-43409e5fd31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:39.248491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8681f45-79c3-4a1e-9068-43409e5fd31a admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:39.248771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8681f45-79c3-4a1e-9068-43409e5fd31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:39.249044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8681f45-79c3-4a1e-9068-43409e5fd31a admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:39.249742 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8681f45-79c3-4a1e-9068-43409e5fd31a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:29:39.250046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8681f45-79c3-4a1e-9068-43409e5fd31a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:39.251655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8681f45-79c3-4a1e-9068-43409e5fd31a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:39.264676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1288/2583] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:29:39.268857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1681d-3eeb-4a5a-b280-13e21666412c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:39.278637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1681d-3eeb-4a5a-b280-13e21666412c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:39.283553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d95031-fcff-44d9-ad48-84c5a7d44a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:39.284137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d95031-fcff-44d9-ad48-84c5a7d44a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:39.284422 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d95031-fcff-44d9-ad48-84c5a7d44a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:39.284696 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d95031-fcff-44d9-ad48-84c5a7d44a41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:39.285356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d95031-fcff-44d9-ad48-84c5a7d44a41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:39.287231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d95031-fcff-44d9-ad48-84c5a7d44a41 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:39.397136 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1296/2584] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:29:39 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:39.401300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8681f45-79c3-4a1e-9068-43409e5fd31a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:39.412040 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8681f45-79c3-4a1e-9068-43409e5fd31a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:39.471747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71734a1d-2cbe-4e83-8df5-7838b1dd010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:39.472398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71734a1d-2cbe-4e83-8df5-7838b1dd010c admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:39.472677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71734a1d-2cbe-4e83-8df5-7838b1dd010c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:39.472951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71734a1d-2cbe-4e83-8df5-7838b1dd010c admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:39.473620 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71734a1d-2cbe-4e83-8df5-7838b1dd010c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:39.475533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71734a1d-2cbe-4e83-8df5-7838b1dd010c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:39.483386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:39.572643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1289/2585] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:29:39 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:39.577271 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d95031-fcff-44d9-ad48-84c5a7d44a41 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:39.587990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d95031-fcff-44d9-ad48-84c5a7d44a41 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:39.592862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4d62f-5226-4cbe-a2b2-1866fe492a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:39.593471 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4d62f-5226-4cbe-a2b2-1866fe492a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:39.593746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4d62f-5226-4cbe-a2b2-1866fe492a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:39.594016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4d62f-5226-4cbe-a2b2-1866fe492a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:39.594719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4d62f-5226-4cbe-a2b2-1866fe492a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:39.596358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4d62f-5226-4cbe-a2b2-1866fe492a97 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:39.606773 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1297/2586] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:29:39.611168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71734a1d-2cbe-4e83-8df5-7838b1dd010c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:39.621781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71734a1d-2cbe-4e83-8df5-7838b1dd010c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:39.626408 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ee25b-f235-4d00-ab22-5ca76adba726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:39.627020 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ee25b-f235-4d00-ab22-5ca76adba726 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:39.627306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ee25b-f235-4d00-ab22-5ca76adba726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:39.627586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ee25b-f235-4d00-ab22-5ca76adba726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:39.628347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ee25b-f235-4d00-ab22-5ca76adba726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:39.630490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ee25b-f235-4d00-ab22-5ca76adba726 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:39.643940 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1290/2587] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:29:39.648149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4d62f-5226-4cbe-a2b2-1866fe492a97 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:39.658681 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4d62f-5226-4cbe-a2b2-1866fe492a97 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:39.663913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7a865-dd0e-4fa9-996a-730bd163b6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:39.664604 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7a865-dd0e-4fa9-996a-730bd163b6df admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:39.664892 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7a865-dd0e-4fa9-996a-730bd163b6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:39.665201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7a865-dd0e-4fa9-996a-730bd163b6df admin admin] PATH_INFO: `/v3/projects/2e1f3b871e254774a05408b816015827/users/f05648c4a067490693bd554f458c6a1b/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:39.680584 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7a865-dd0e-4fa9-996a-730bd163b6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1f3b871e254774a05408b816015827, user_id=f05648c4a067490693bd554f458c6a1b, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:39.684323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7a865-dd0e-4fa9-996a-730bd163b6df admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:39.754474 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1298/2588] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:29:39 2022] PUT /identity/v3/projects/2e1f3b871e254774a05408b816015827/users/f05648c4a067490693bd554f458c6a1b/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:39.759530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6b6da-3687-4b42-aa87-405c95dfcfe6 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:39.759837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6b6da-3687-4b42-aa87-405c95dfcfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:39.760218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6b6da-3687-4b42-aa87-405c95dfcfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:39.779533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8cb6b6da-3687-4b42-aa87-405c95dfcfe6 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:39.780761 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8cb6b6da-3687-4b42-aa87-405c95dfcfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:39.846766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1291/2589] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:39.856782 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7a865-dd0e-4fa9-996a-730bd163b6df admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:39.858475 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-dba7a865-dd0e-4fa9-996a-730bd163b6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:39.915554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7a865-dd0e-4fa9-996a-730bd163b6df admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:39.919731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8a398-a8f0-4ace-8d2d-745a6a4d7f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:39.920404 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8a398-a8f0-4ace-8d2d-745a6a4d7f43 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:39.920687 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8a398-a8f0-4ace-8d2d-745a6a4d7f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:39.920951 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8a398-a8f0-4ace-8d2d-745a6a4d7f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:39.930485 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8a398-a8f0-4ace-8d2d-745a6a4d7f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:39.931936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8a398-a8f0-4ace-8d2d-745a6a4d7f43 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:39.941877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1299/2590] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:45.934240 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e3b67-0bdf-4214-9720-2d1b86d86c28 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:45.935851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e3b67-0bdf-4214-9720-2d1b86d86c28 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:45.935851 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e3b67-0bdf-4214-9720-2d1b86d86c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:45.973349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-be0e3b67-0bdf-4214-9720-2d1b86d86c28 None None] MFA Rules not processed for user `f05648c4a067490693bd554f458c6a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:45.974651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-be0e3b67-0bdf-4214-9720-2d1b86d86c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:46.083846 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1292/2591] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:46.099955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8a398-a8f0-4ace-8d2d-745a6a4d7f43 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:46.101712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-30e8a398-a8f0-4ace-8d2d-745a6a4d7f43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:46.204188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8a398-a8f0-4ace-8d2d-745a6a4d7f43 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:46.213206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594a0e2-a193-4985-8340-f8fb2b5a1491 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:46.213888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a0e2-a193-4985-8340-f8fb2b5a1491 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:46.216489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a0e2-a193-4985-8340-f8fb2b5a1491 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:46.216489 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594a0e2-a193-4985-8340-f8fb2b5a1491 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:46.224061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594a0e2-a193-4985-8340-f8fb2b5a1491 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:46.226261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594a0e2-a193-4985-8340-f8fb2b5a1491 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:46.240046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1300/2592] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:49.322396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428fee4-0d4e-4bc8-942c-8c79abfacd88 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:49.322966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428fee4-0d4e-4bc8-942c-8c79abfacd88 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:49.322966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428fee4-0d4e-4bc8-942c-8c79abfacd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:49.343170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c428fee4-0d4e-4bc8-942c-8c79abfacd88 None None] MFA Rules not processed for user `f05648c4a067490693bd554f458c6a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:49.344427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c428fee4-0d4e-4bc8-942c-8c79abfacd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:49.351432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1293/2593] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:49.686889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594a0e2-a193-4985-8340-f8fb2b5a1491 service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:49.696588 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:49.701348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594a0e2-a193-4985-8340-f8fb2b5a1491 service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:49.707157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c6f5e-b0d5-41e8-9abe-efaf4e702a6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:49.707987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c6f5e-b0d5-41e8-9abe-efaf4e702a6b service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:49.708488 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c6f5e-b0d5-41e8-9abe-efaf4e702a6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:49.708957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c6f5e-b0d5-41e8-9abe-efaf4e702a6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:49.712030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bf3c6f5e-b0d5-41e8-9abe-efaf4e702a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:49.723837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c6f5e-b0d5-41e8-9abe-efaf4e702a6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:49.725532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c6f5e-b0d5-41e8-9abe-efaf4e702a6b service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:49.735277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1301/2594] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:50.659897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428fee4-0d4e-4bc8-942c-8c79abfacd88 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:50.670087 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428fee4-0d4e-4bc8-942c-8c79abfacd88 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:50.673644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5cd89-ebc1-4977-9f71-190ff5be9911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:50.675384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5cd89-ebc1-4977-9f71-190ff5be9911 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:50.675384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5cd89-ebc1-4977-9f71-190ff5be9911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:50.675384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5cd89-ebc1-4977-9f71-190ff5be9911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:50.676581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-82f5cd89-ebc1-4977-9f71-190ff5be9911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:50.688718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5cd89-ebc1-4977-9f71-190ff5be9911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:50.690134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5cd89-ebc1-4977-9f71-190ff5be9911 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:50.701447 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1294/2595] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:54.616170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c6f5e-b0d5-41e8-9abe-efaf4e702a6b service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:54.618631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bf3c6f5e-b0d5-41e8-9abe-efaf4e702a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:54.692725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c6f5e-b0d5-41e8-9abe-efaf4e702a6b service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:54.697929 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd4a4-8998-42d2-841c-6062c709016c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:54.699117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd4a4-8998-42d2-841c-6062c709016c service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:54.699593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd4a4-8998-42d2-841c-6062c709016c service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:54.699906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fd4a4-8998-42d2-841c-6062c709016c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:54.701756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-363fd4a4-8998-42d2-841c-6062c709016c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:54.713503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fd4a4-8998-42d2-841c-6062c709016c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:54.715146 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fd4a4-8998-42d2-841c-6062c709016c service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:54.725174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1302/2596] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:54.744689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5cd89-ebc1-4977-9f71-190ff5be9911 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:54.746790 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-82f5cd89-ebc1-4977-9f71-190ff5be9911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:54.768686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5cd89-ebc1-4977-9f71-190ff5be9911 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:54.770435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:54.772224 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:54.772453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:54.772934 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] PATH_INFO: `/v3/users/2d417104bf4040d4ae9ccb08097b5255` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:54.775642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d417104bf4040d4ae9ccb08097b5255)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:54.779599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:55.015016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:INSERT=3,keystone:SELECT=91,keystone:DELETE=6 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:55.018013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d417104bf4040d4ae9ccb08097b5255'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:55.025180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d417104bf4040d4ae9ccb08097b5255'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:55.117789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d417104bf4040d4ae9ccb08097b5255'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:55.119229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d417104bf4040d4ae9ccb08097b5255'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:55.119881 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f4f386ed-8d66-4e6d-99e4-12faf37636a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d417104bf4040d4ae9ccb08097b5255'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:55.139840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1295/2597] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:29:54 2022] DELETE /identity/v3/users/2d417104bf4040d4ae9ccb08097b5255 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:29:55.434306 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd4a4-8998-42d2-841c-6062c709016c service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:55.445613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fd4a4-8998-42d2-841c-6062c709016c service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:55.502725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:55.503363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:55.503646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:55.503958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] PATH_INFO: `/v3/projects/1ce28e8fffc1495cba5caca6092a3318` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:55.505100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce28e8fffc1495cba5caca6092a3318)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:55.506993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:55.606987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce28e8fffc1495cba5caca6092a3318', 'request_id': 'req-4c318472-5f6b-42dc-a0a6-f230b7f08201'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:55.613793 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce28e8fffc1495cba5caca6092a3318', 'request_id': 'req-4c318472-5f6b-42dc-a0a6-f230b7f08201'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:55.619926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce28e8fffc1495cba5caca6092a3318', 'request_id': 'req-4c318472-5f6b-42dc-a0a6-f230b7f08201'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:55.689308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] The token cache is being invalidate because project 1ce28e8fffc1495cba5caca6092a3318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:29:55.689308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:55.689308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4c318472-5f6b-42dc-a0a6-f230b7f08201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:55.689308 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1303/2598] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:29:55 2022] DELETE /identity/v3/projects/1ce28e8fffc1495cba5caca6092a3318 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:55.693644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f57e0-a0cf-4e81-b1f6-a4d15658cd7a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:55.693810 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f57e0-a0cf-4e81-b1f6-a4d15658cd7a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:55.694048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f57e0-a0cf-4e81-b1f6-a4d15658cd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:55.715175 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-737f57e0-a0cf-4e81-b1f6-a4d15658cd7a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:55.716503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-737f57e0-a0cf-4e81-b1f6-a4d15658cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:55.779840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1296/2599] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:55.789054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82640273-7f22-46fb-be22-83510241aa27 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:55.789358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82640273-7f22-46fb-be22-83510241aa27 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:55.789638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82640273-7f22-46fb-be22-83510241aa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:55.812059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-82640273-7f22-46fb-be22-83510241aa27 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:55.813335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-82640273-7f22-46fb-be22-83510241aa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:55.821344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1304/2600] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:55.826143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f57e0-a0cf-4e81-b1f6-a4d15658cd7a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:55.835700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f57e0-a0cf-4e81-b1f6-a4d15658cd7a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:55.841173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c2f01-656e-48ca-8757-f8a7fcccb4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:55.841788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c2f01-656e-48ca-8757-f8a7fcccb4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:55.842071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c2f01-656e-48ca-8757-f8a7fcccb4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:55.842344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c2f01-656e-48ca-8757-f8a7fcccb4d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:55.843118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c2f01-656e-48ca-8757-f8a7fcccb4d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:29:55.843424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c2f01-656e-48ca-8757-f8a7fcccb4d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:55.845170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c2f01-656e-48ca-8757-f8a7fcccb4d4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:55.856894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1297/2601] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:29:55.861587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82640273-7f22-46fb-be22-83510241aa27 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:55.867289 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:55.871125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82640273-7f22-46fb-be22-83510241aa27 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:55.875934 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8cd9d-491f-4ce2-9efe-ade44f5bd048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:55.876536 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8cd9d-491f-4ce2-9efe-ade44f5bd048 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:55.876821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8cd9d-491f-4ce2-9efe-ade44f5bd048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:55.877102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8cd9d-491f-4ce2-9efe-ade44f5bd048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:55.877773 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8cd9d-491f-4ce2-9efe-ade44f5bd048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:55.879728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8cd9d-491f-4ce2-9efe-ade44f5bd048 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:55.979676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1305/2602] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:29:55 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:55.984477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c2f01-656e-48ca-8757-f8a7fcccb4d4 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:55.994759 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c2f01-656e-48ca-8757-f8a7fcccb4d4 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:56.053615 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b72e5-cdae-40b7-a1bd-435339882de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:56.054839 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b72e5-cdae-40b7-a1bd-435339882de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:56.055128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b72e5-cdae-40b7-a1bd-435339882de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:56.055504 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b72e5-cdae-40b7-a1bd-435339882de7 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:56.056205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b72e5-cdae-40b7-a1bd-435339882de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:56.058700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b72e5-cdae-40b7-a1bd-435339882de7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:56.149314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1298/2603] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:29:55 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:29:56.153531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8cd9d-491f-4ce2-9efe-ade44f5bd048 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:56.163985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8cd9d-491f-4ce2-9efe-ade44f5bd048 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:56.168705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd538f7e-91f5-4c92-80a1-24edff2f5af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:56.169315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd538f7e-91f5-4c92-80a1-24edff2f5af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:56.169603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd538f7e-91f5-4c92-80a1-24edff2f5af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:56.169881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd538f7e-91f5-4c92-80a1-24edff2f5af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:56.170654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd538f7e-91f5-4c92-80a1-24edff2f5af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:56.172296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd538f7e-91f5-4c92-80a1-24edff2f5af2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:56.182088 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1306/2604] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:29:56.186413 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b72e5-cdae-40b7-a1bd-435339882de7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:56.196365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b72e5-cdae-40b7-a1bd-435339882de7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:56.201815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897388f-a761-4544-90a8-952876bbfff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:56.202572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897388f-a761-4544-90a8-952876bbfff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:56.202853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897388f-a761-4544-90a8-952876bbfff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:56.203128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897388f-a761-4544-90a8-952876bbfff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:56.204026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897388f-a761-4544-90a8-952876bbfff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:56.205637 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897388f-a761-4544-90a8-952876bbfff4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:56.217622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1299/2605] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:29:56.222445 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd538f7e-91f5-4c92-80a1-24edff2f5af2 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:56.232311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd538f7e-91f5-4c92-80a1-24edff2f5af2 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:56.237227 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100b9fe-d625-4aaf-bd20-dc18b4fb0f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:56.237883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100b9fe-d625-4aaf-bd20-dc18b4fb0f7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:56.238168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100b9fe-d625-4aaf-bd20-dc18b4fb0f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:56.238448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100b9fe-d625-4aaf-bd20-dc18b4fb0f7d admin admin] PATH_INFO: `/v3/projects/d856b3b71eda4347b868e5b74fbd6df2/users/ad4b53e85af7468faf1d433bc742a9ea/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:56.253741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100b9fe-d625-4aaf-bd20-dc18b4fb0f7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d856b3b71eda4347b868e5b74fbd6df2, user_id=ad4b53e85af7468faf1d433bc742a9ea, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:56.256520 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100b9fe-d625-4aaf-bd20-dc18b4fb0f7d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:56.327711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1307/2606] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:29:56 2022] PUT /identity/v3/projects/d856b3b71eda4347b868e5b74fbd6df2/users/ad4b53e85af7468faf1d433bc742a9ea/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:56.338106 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa14ef-c564-480f-b434-8fd536979675 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:56.338400 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa14ef-c564-480f-b434-8fd536979675 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:56.338676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa14ef-c564-480f-b434-8fd536979675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:56.366691 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3afa14ef-c564-480f-b434-8fd536979675 None None] MFA Rules not processed for user `ad4b53e85af7468faf1d433bc742a9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:56.367998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3afa14ef-c564-480f-b434-8fd536979675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:56.383598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:56.435329 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1300/2607] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:56.447328 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100b9fe-d625-4aaf-bd20-dc18b4fb0f7d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:56.449094 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5100b9fe-d625-4aaf-bd20-dc18b4fb0f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:56.521277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100b9fe-d625-4aaf-bd20-dc18b4fb0f7d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:56.526804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff78e0-2ef8-48c2-b60b-02c97e5ffeb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:56.527409 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff78e0-2ef8-48c2-b60b-02c97e5ffeb5 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:56.527746 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff78e0-2ef8-48c2-b60b-02c97e5ffeb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:56.528035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff78e0-2ef8-48c2-b60b-02c97e5ffeb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:56.535517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff78e0-2ef8-48c2-b60b-02c97e5ffeb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:56.537097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff78e0-2ef8-48c2-b60b-02c97e5ffeb5 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:56.546311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1308/2608] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:29:59.391070 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa14ef-c564-480f-b434-8fd536979675 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:59.392526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3afa14ef-c564-480f-b434-8fd536979675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:59.458393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa14ef-c564-480f-b434-8fd536979675 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:59.462906 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:59.463549 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:59.463860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:59.464160 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] PATH_INFO: `/v3/users/07ff90c4e586488eaa13365fbf7cd339` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:59.465462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ff90c4e586488eaa13365fbf7cd339)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:59.467443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:59.618582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ff90c4e586488eaa13365fbf7cd339'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:59.626259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ff90c4e586488eaa13365fbf7cd339'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:59.686232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ff90c4e586488eaa13365fbf7cd339'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:59.686849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ff90c4e586488eaa13365fbf7cd339'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:59.687479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ff90c4e586488eaa13365fbf7cd339'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:29:59.705155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1301/2609] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:29:59 2022] DELETE /identity/v3/users/07ff90c4e586488eaa13365fbf7cd339 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:29:59.709378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1b637-77dd-4701-a057-78d5ae09eb6c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:59.709841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1b637-77dd-4701-a057-78d5ae09eb6c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:59.709960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1b637-77dd-4701-a057-78d5ae09eb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:59.728922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-31e1b637-77dd-4701-a057-78d5ae09eb6c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:29:59.730847 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-31e1b637-77dd-4701-a057-78d5ae09eb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:59.789202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1309/2610] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:29:59.799399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:29:59.800869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:29:59.839764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:29:59.857011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e08e92-bddd-4936-8924-a8dc6fca18cd admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:29:59.860939 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c95c9-4c01-49f2-8b04-5018f3bc31ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:29:59.861558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c95c9-4c01-49f2-8b04-5018f3bc31ea service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:29:59.861842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c95c9-4c01-49f2-8b04-5018f3bc31ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:29:59.862120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c95c9-4c01-49f2-8b04-5018f3bc31ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:29:59.871707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c95c9-4c01-49f2-8b04-5018f3bc31ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:29:59.873268 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c95c9-4c01-49f2-8b04-5018f3bc31ea service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:29:59.882588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1302/2611] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:00.279689 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1b637-77dd-4701-a057-78d5ae09eb6c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:00.281193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-31e1b637-77dd-4701-a057-78d5ae09eb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:00.295178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1b637-77dd-4701-a057-78d5ae09eb6c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:00.299825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:00.305744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:00.305744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:00.305744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] PATH_INFO: `/v3/projects/bd630b3de83e462e82ccdc98d2413842` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:00.305744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd630b3de83e462e82ccdc98d2413842)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:00.305744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:00.417676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:00.430784 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c95c9-4c01-49f2-8b04-5018f3bc31ea service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:00.432363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-bd6c95c9-4c01-49f2-8b04-5018f3bc31ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:00.437398 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd630b3de83e462e82ccdc98d2413842', 'request_id': 'req-2e6048e5-6d47-4573-9859-367478153ab8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:00.443867 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd630b3de83e462e82ccdc98d2413842', 'request_id': 'req-2e6048e5-6d47-4573-9859-367478153ab8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:00.444996 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c95c9-4c01-49f2-8b04-5018f3bc31ea service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:00.450473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd630b3de83e462e82ccdc98d2413842', 'request_id': 'req-2e6048e5-6d47-4573-9859-367478153ab8'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:00.494169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44abbe-1461-42d3-bd0f-754fcc169d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:00.494783 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44abbe-1461-42d3-bd0f-754fcc169d76 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:00.495060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44abbe-1461-42d3-bd0f-754fcc169d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:00.495950 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44abbe-1461-42d3-bd0f-754fcc169d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:00.497713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6c44abbe-1461-42d3-bd0f-754fcc169d76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:00.547835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] The token cache is being invalidate because project bd630b3de83e462e82ccdc98d2413842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:00.548425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:00.549083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2e6048e5-6d47-4573-9859-367478153ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:00.550956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1310/2612] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:00 2022] DELETE /identity/v3/projects/bd630b3de83e462e82ccdc98d2413842 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:00.555481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c66ce-2357-4346-b1d1-82c8e8613e76 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:00.558357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c66ce-2357-4346-b1d1-82c8e8613e76 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:00.558357 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c66ce-2357-4346-b1d1-82c8e8613e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:00.568021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44abbe-1461-42d3-bd0f-754fcc169d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:00.569465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44abbe-1461-42d3-bd0f-754fcc169d76 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:00.576143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a83c66ce-2357-4346-b1d1-82c8e8613e76 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:00.577351 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a83c66ce-2357-4346-b1d1-82c8e8613e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:00.578805 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1303/2613] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:30:00.657532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1311/2614] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:30:00.659318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c968-fb43-449d-af5e-eb6f1ecd5e7a None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:00.659608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c968-fb43-449d-af5e-eb6f1ecd5e7a None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:00.659927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c968-fb43-449d-af5e-eb6f1ecd5e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:00.682507 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-9882c968-fb43-449d-af5e-eb6f1ecd5e7a None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:00.685126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-9882c968-fb43-449d-af5e-eb6f1ecd5e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:00.694126 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1304/2615] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:30:00.698353 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c66ce-2357-4346-b1d1-82c8e8613e76 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:00.708016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c66ce-2357-4346-b1d1-82c8e8613e76 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:00.713195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5595a-c335-46b6-bdcb-38dcc68ac554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:00.713789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5595a-c335-46b6-bdcb-38dcc68ac554 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:00.714701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5595a-c335-46b6-bdcb-38dcc68ac554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:00.715040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5595a-c335-46b6-bdcb-38dcc68ac554 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:00.715835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5595a-c335-46b6-bdcb-38dcc68ac554 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:30:00.716217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5595a-c335-46b6-bdcb-38dcc68ac554 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:00.717898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5595a-c335-46b6-bdcb-38dcc68ac554 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:00.730904 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1312/2616] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:30:00.735573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882c968-fb43-449d-af5e-eb6f1ecd5e7a None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:00.745017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882c968-fb43-449d-af5e-eb6f1ecd5e7a None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:00.749966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a2fb5-4744-45d7-997c-e1d6f6ccf0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:00.750692 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a2fb5-4744-45d7-997c-e1d6f6ccf0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:00.751039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a2fb5-4744-45d7-997c-e1d6f6ccf0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:00.751420 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a2fb5-4744-45d7-997c-e1d6f6ccf0d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:00.752205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a2fb5-4744-45d7-997c-e1d6f6ccf0d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:00.754190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a2fb5-4744-45d7-997c-e1d6f6ccf0d6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:00.881979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1305/2617] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:00 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:00.886805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5595a-c335-46b6-bdcb-38dcc68ac554 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:00.896989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5595a-c335-46b6-bdcb-38dcc68ac554 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:00.954622 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e36875-736d-4ff2-8b2e-ad784d6f005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:00.955249 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e36875-736d-4ff2-8b2e-ad784d6f005b admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:00.955531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e36875-736d-4ff2-8b2e-ad784d6f005b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:00.955838 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e36875-736d-4ff2-8b2e-ad784d6f005b admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:00.956580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e36875-736d-4ff2-8b2e-ad784d6f005b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:00.958537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e36875-736d-4ff2-8b2e-ad784d6f005b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:01.092802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1313/2618] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:00 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:01.095916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a2fb5-4744-45d7-997c-e1d6f6ccf0d6 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:01.108018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a2fb5-4744-45d7-997c-e1d6f6ccf0d6 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:01.117396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6d510-0d8a-46f1-8a96-629bf8774bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:01.121582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6d510-0d8a-46f1-8a96-629bf8774bbe admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:01.121582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6d510-0d8a-46f1-8a96-629bf8774bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:01.121582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6d510-0d8a-46f1-8a96-629bf8774bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:01.121582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6d510-0d8a-46f1-8a96-629bf8774bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:01.121582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6d510-0d8a-46f1-8a96-629bf8774bbe admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:01.134633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1306/2619] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:30:01.136863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e36875-736d-4ff2-8b2e-ad784d6f005b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:01.148456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e36875-736d-4ff2-8b2e-ad784d6f005b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:01.154121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e133638-5126-4028-a9c5-14eeadc79c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:01.154906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133638-5126-4028-a9c5-14eeadc79c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:01.155193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133638-5126-4028-a9c5-14eeadc79c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:01.155465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133638-5126-4028-a9c5-14eeadc79c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:01.156209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e133638-5126-4028-a9c5-14eeadc79c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:01.157809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e133638-5126-4028-a9c5-14eeadc79c14 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:01.169605 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1314/2620] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:01.174098 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6d510-0d8a-46f1-8a96-629bf8774bbe admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:01.185341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6d510-0d8a-46f1-8a96-629bf8774bbe admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:01.190098 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa7e1d-aa19-4826-93f8-281e5d8529ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:01.190843 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa7e1d-aa19-4826-93f8-281e5d8529ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:01.191138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa7e1d-aa19-4826-93f8-281e5d8529ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:01.191409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa7e1d-aa19-4826-93f8-281e5d8529ab admin admin] PATH_INFO: `/v3/projects/10e81968971c4c3fa793ca081d168209/users/165812ae8a304c1a94bc62dd84144d3c/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:01.210458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa7e1d-aa19-4826-93f8-281e5d8529ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e81968971c4c3fa793ca081d168209, user_id=165812ae8a304c1a94bc62dd84144d3c, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:01.213322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa7e1d-aa19-4826-93f8-281e5d8529ab admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:01.344227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1307/2621] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:01 2022] PUT /identity/v3/projects/10e81968971c4c3fa793ca081d168209/users/165812ae8a304c1a94bc62dd84144d3c/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:01.349651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a476b39-f481-423b-a9f4-828c2be82dfd None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:01.350048 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a476b39-f481-423b-a9f4-828c2be82dfd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:01.350392 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a476b39-f481-423b-a9f4-828c2be82dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:01.370283 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-1a476b39-f481-423b-a9f4-828c2be82dfd None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:01.372058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1a476b39-f481-423b-a9f4-828c2be82dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:01.390302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:01.432653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1315/2622] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:01.442903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa7e1d-aa19-4826-93f8-281e5d8529ab admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:01.444636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c8fa7e1d-aa19-4826-93f8-281e5d8529ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:01.477774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:01.503680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa7e1d-aa19-4826-93f8-281e5d8529ab admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:01.507344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7402f2-cbf5-406d-bcad-a4025144ec10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:01.508062 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7402f2-cbf5-406d-bcad-a4025144ec10 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:01.508347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7402f2-cbf5-406d-bcad-a4025144ec10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:01.508626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7402f2-cbf5-406d-bcad-a4025144ec10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:01.518525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7402f2-cbf5-406d-bcad-a4025144ec10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:01.520004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7402f2-cbf5-406d-bcad-a4025144ec10 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:01.529801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1308/2623] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:02.838937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49adf37-9fa9-4887-88e9-3675f671a0e2 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:02.840169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49adf37-9fa9-4887-88e9-3675f671a0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:02.840169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49adf37-9fa9-4887-88e9-3675f671a0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:02.877850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-d49adf37-9fa9-4887-88e9-3675f671a0e2 None None] MFA Rules not processed for user `165812ae8a304c1a94bc62dd84144d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:02.879205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d49adf37-9fa9-4887-88e9-3675f671a0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:02.945369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1316/2624] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:02.956862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7402f2-cbf5-406d-bcad-a4025144ec10 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:02.958274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-af7402f2-cbf5-406d-bcad-a4025144ec10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:03.015171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7402f2-cbf5-406d-bcad-a4025144ec10 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:03.018733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818499c-dda0-4b37-a7a1-6b3f802ab081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:03.019347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818499c-dda0-4b37-a7a1-6b3f802ab081 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:03.019629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818499c-dda0-4b37-a7a1-6b3f802ab081 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:03.019947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818499c-dda0-4b37-a7a1-6b3f802ab081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:03.028107 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818499c-dda0-4b37-a7a1-6b3f802ab081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:03.029573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818499c-dda0-4b37-a7a1-6b3f802ab081 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:03.038455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1309/2625] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:30:04.837523 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf749cc-18a7-4368-bdb5-ac91a5c85500 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:04.838379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf749cc-18a7-4368-bdb5-ac91a5c85500 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:04.838809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf749cc-18a7-4368-bdb5-ac91a5c85500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:04.874390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fbf749cc-18a7-4368-bdb5-ac91a5c85500 None None] MFA Rules not processed for user `165812ae8a304c1a94bc62dd84144d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:04.877919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fbf749cc-18a7-4368-bdb5-ac91a5c85500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:04.887384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818499c-dda0-4b37-a7a1-6b3f802ab081 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:04.889293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6818499c-dda0-4b37-a7a1-6b3f802ab081 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:04.899888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1317/2626] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:04.927281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818499c-dda0-4b37-a7a1-6b3f802ab081 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:04.936674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:04.937642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:04.938144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:04.938684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] PATH_INFO: `/v3/users/ad4b53e85af7468faf1d433bc742a9ea` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:04.940182 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad4b53e85af7468faf1d433bc742a9ea)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:04.943963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:05.162188 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad4b53e85af7468faf1d433bc742a9ea'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:05.171346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad4b53e85af7468faf1d433bc742a9ea'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:05.253859 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad4b53e85af7468faf1d433bc742a9ea'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:05.254912 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad4b53e85af7468faf1d433bc742a9ea'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:05.255765 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad4b53e85af7468faf1d433bc742a9ea'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:05.276561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1310/2627] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:04 2022] DELETE /identity/v3/users/ad4b53e85af7468faf1d433bc742a9ea => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:05.281684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aff977-720f-4098-ba28-035aae888c0e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:05.281929 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aff977-720f-4098-ba28-035aae888c0e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:05.285036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aff977-720f-4098-ba28-035aae888c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:05.303692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b4aff977-720f-4098-ba28-035aae888c0e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:05.307738 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4aff977-720f-4098-ba28-035aae888c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:05.366483 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1318/2628] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:30:05.376611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:05.378096 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:05.453589 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:05.460897 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157b899-d687-46c3-90b8-bb4941ed38a8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:05.464770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c729df-1338-45fd-ab3a-d72ce8c7ad75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:05.465942 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c729df-1338-45fd-ab3a-d72ce8c7ad75 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:05.466234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c729df-1338-45fd-ab3a-d72ce8c7ad75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:05.466512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c729df-1338-45fd-ab3a-d72ce8c7ad75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:05.486602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c729df-1338-45fd-ab3a-d72ce8c7ad75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:05.488120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c729df-1338-45fd-ab3a-d72ce8c7ad75 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:05.499572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1311/2629] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:06.077830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aff977-720f-4098-ba28-035aae888c0e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:06.079460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4aff977-720f-4098-ba28-035aae888c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:06.106521 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aff977-720f-4098-ba28-035aae888c0e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:06.108249 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:06.108249 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:06.108441 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:06.109734 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] PATH_INFO: `/v3/projects/d856b3b71eda4347b868e5b74fbd6df2` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:06.111389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d856b3b71eda4347b868e5b74fbd6df2)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:06.113330 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:06.254585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:06.261113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd856b3b71eda4347b868e5b74fbd6df2', 'request_id': 'req-bffdae02-d6f4-4b56-b250-f96f162e6754'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:06.267806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd856b3b71eda4347b868e5b74fbd6df2', 'request_id': 'req-bffdae02-d6f4-4b56-b250-f96f162e6754'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:06.273972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd856b3b71eda4347b868e5b74fbd6df2', 'request_id': 'req-bffdae02-d6f4-4b56-b250-f96f162e6754'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:06.388801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] The token cache is being invalidate because project d856b3b71eda4347b868e5b74fbd6df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:06.388801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:06.388801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bffdae02-d6f4-4b56-b250-f96f162e6754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:06.389390 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1319/2630] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:06 2022] DELETE /identity/v3/projects/d856b3b71eda4347b868e5b74fbd6df2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:06.394609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6778a-979b-4fef-8adb-ef6df25ae49d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:06.395114 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6778a-979b-4fef-8adb-ef6df25ae49d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:06.395364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6778a-979b-4fef-8adb-ef6df25ae49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:06.418862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-19c6778a-979b-4fef-8adb-ef6df25ae49d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:06.420143 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-19c6778a-979b-4fef-8adb-ef6df25ae49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:06.496141 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1312/2631] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:06.504274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1709d-bc8b-4f58-97bc-30bf32eb5326 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:06.504859 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1709d-bc8b-4f58-97bc-30bf32eb5326 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:06.505487 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1709d-bc8b-4f58-97bc-30bf32eb5326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:06.528436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2ae1709d-bc8b-4f58-97bc-30bf32eb5326 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:06.530104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2ae1709d-bc8b-4f58-97bc-30bf32eb5326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:06.539651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1320/2632] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:30:06.544881 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6778a-979b-4fef-8adb-ef6df25ae49d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:06.559265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6778a-979b-4fef-8adb-ef6df25ae49d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:06.566048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c468ed-e518-4389-abf5-ac9496e9aeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:06.566772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c468ed-e518-4389-abf5-ac9496e9aeae admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:06.567124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c468ed-e518-4389-abf5-ac9496e9aeae admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:06.567468 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c468ed-e518-4389-abf5-ac9496e9aeae admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:06.568275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c468ed-e518-4389-abf5-ac9496e9aeae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:30:06.568660 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c468ed-e518-4389-abf5-ac9496e9aeae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:06.570380 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c468ed-e518-4389-abf5-ac9496e9aeae admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:06.588688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1313/2633] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:30:06.590866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1709d-bc8b-4f58-97bc-30bf32eb5326 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:06.607570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1709d-bc8b-4f58-97bc-30bf32eb5326 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:06.615812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d3ad1-bb1e-4620-8697-aef05d10ff85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:06.616545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d3ad1-bb1e-4620-8697-aef05d10ff85 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:06.616850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d3ad1-bb1e-4620-8697-aef05d10ff85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:06.617128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d3ad1-bb1e-4620-8697-aef05d10ff85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:06.617816 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d3ad1-bb1e-4620-8697-aef05d10ff85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:06.620199 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d3ad1-bb1e-4620-8697-aef05d10ff85 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:06.720996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1321/2634] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:06 2022] POST /identity/v3/projects => generated 403 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:06.726029 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c468ed-e518-4389-abf5-ac9496e9aeae admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:06.739177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c468ed-e518-4389-abf5-ac9496e9aeae admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:06.805723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9db647-c593-4743-aa3c-ef9b3a2cfffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:06.806368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9db647-c593-4743-aa3c-ef9b3a2cfffb admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:06.806651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9db647-c593-4743-aa3c-ef9b3a2cfffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:06.810781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9db647-c593-4743-aa3c-ef9b3a2cfffb admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:06.810781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9db647-c593-4743-aa3c-ef9b3a2cfffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:06.810781 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9db647-c593-4743-aa3c-ef9b3a2cfffb admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:06.925274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1314/2635] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:06 2022] POST /identity/v3/users => generated 302 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:06.930148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d3ad1-bb1e-4620-8697-aef05d10ff85 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:06.956458 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d3ad1-bb1e-4620-8697-aef05d10ff85 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:06.961911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2383e-46db-4dc4-8588-ab63bf3d9882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:06.962561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2383e-46db-4dc4-8588-ab63bf3d9882 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:06.962828 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2383e-46db-4dc4-8588-ab63bf3d9882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:06.963220 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2383e-46db-4dc4-8588-ab63bf3d9882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:06.963926 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2383e-46db-4dc4-8588-ab63bf3d9882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:06.966127 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2383e-46db-4dc4-8588-ab63bf3d9882 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:06.976915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1322/2636] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:06.981211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9db647-c593-4743-aa3c-ef9b3a2cfffb admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:06.995187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9db647-c593-4743-aa3c-ef9b3a2cfffb admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:07.010355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622ad28-cece-45da-af34-4fa06ffd7476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:07.011008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622ad28-cece-45da-af34-4fa06ffd7476 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:07.011301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622ad28-cece-45da-af34-4fa06ffd7476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:07.011602 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622ad28-cece-45da-af34-4fa06ffd7476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:07.012873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622ad28-cece-45da-af34-4fa06ffd7476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:07.015614 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622ad28-cece-45da-af34-4fa06ffd7476 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:07.030659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1315/2637] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:07.034965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2383e-46db-4dc4-8588-ab63bf3d9882 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:07.045871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2383e-46db-4dc4-8588-ab63bf3d9882 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:07.052443 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713773a4-6d63-4a68-955f-48c1fde537f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:07.053640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713773a4-6d63-4a68-955f-48c1fde537f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:07.053640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713773a4-6d63-4a68-955f-48c1fde537f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:07.053857 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713773a4-6d63-4a68-955f-48c1fde537f8 admin admin] PATH_INFO: `/v3/projects/fb2e18e3fb414d12b9361c0dd9089d00/users/d82edd60e65f45528f9f56d82797b9fb/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:07.079988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713773a4-6d63-4a68-955f-48c1fde537f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb2e18e3fb414d12b9361c0dd9089d00, user_id=d82edd60e65f45528f9f56d82797b9fb, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:07.081122 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713773a4-6d63-4a68-955f-48c1fde537f8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:07.194613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1323/2638] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:07 2022] PUT /identity/v3/projects/fb2e18e3fb414d12b9361c0dd9089d00/users/d82edd60e65f45528f9f56d82797b9fb/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:07.204258 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3201f-15f0-47e5-bc64-d6160e662b0d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:07.204584 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3201f-15f0-47e5-bc64-d6160e662b0d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:07.204867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3201f-15f0-47e5-bc64-d6160e662b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:07.240466 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-0dc3201f-15f0-47e5-bc64-d6160e662b0d None None] MFA Rules not processed for user `d82edd60e65f45528f9f56d82797b9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:07.241043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0dc3201f-15f0-47e5-bc64-d6160e662b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:07.302723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:07.328124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1316/2639] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:07 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:07.332758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713773a4-6d63-4a68-955f-48c1fde537f8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:07.400282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713773a4-6d63-4a68-955f-48c1fde537f8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:07.404965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68839a08-c1ad-4333-8a10-8e8ddff6494e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:07.405623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68839a08-c1ad-4333-8a10-8e8ddff6494e admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:07.405903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68839a08-c1ad-4333-8a10-8e8ddff6494e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:07.406168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68839a08-c1ad-4333-8a10-8e8ddff6494e admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:07.406858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68839a08-c1ad-4333-8a10-8e8ddff6494e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:07.408786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68839a08-c1ad-4333-8a10-8e8ddff6494e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:07.521893 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1324/2640] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:07 2022] POST /identity/v3/projects => generated 403 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:30:07.526085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3201f-15f0-47e5-bc64-d6160e662b0d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:07.537153 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3201f-15f0-47e5-bc64-d6160e662b0d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:07.610038 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056030d-6e05-40dc-945b-56de475cdd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:07.610789 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056030d-6e05-40dc-945b-56de475cdd21 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:07.611063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056030d-6e05-40dc-945b-56de475cdd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:07.611329 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056030d-6e05-40dc-945b-56de475cdd21 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:07.622123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056030d-6e05-40dc-945b-56de475cdd21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:07.622123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056030d-6e05-40dc-945b-56de475cdd21 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:07.715125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1317/2641] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:07 2022] POST /identity/v3/users => generated 308 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:07.719243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68839a08-c1ad-4333-8a10-8e8ddff6494e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:07.732564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68839a08-c1ad-4333-8a10-8e8ddff6494e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:07.737607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a5187-b750-439f-9a72-f8bb9c3ac3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:07.738232 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a5187-b750-439f-9a72-f8bb9c3ac3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:07.738518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a5187-b750-439f-9a72-f8bb9c3ac3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:07.738799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a5187-b750-439f-9a72-f8bb9c3ac3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:07.739503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a5187-b750-439f-9a72-f8bb9c3ac3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:07.741152 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a5187-b750-439f-9a72-f8bb9c3ac3e0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:07.754805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1325/2642] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:30:07.759145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056030d-6e05-40dc-945b-56de475cdd21 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:07.769333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056030d-6e05-40dc-945b-56de475cdd21 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:07.774090 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdd8c7-7ad1-42d9-aa08-882334260d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:07.774685 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdd8c7-7ad1-42d9-aa08-882334260d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:07.774965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdd8c7-7ad1-42d9-aa08-882334260d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:07.775243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdd8c7-7ad1-42d9-aa08-882334260d5d admin admin] PATH_INFO: `/v3/projects/a83ea779112a4f9d8afbfe3fcec0ea08/users/5696d1c26bdf4324be3ff56176991860/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:07.789953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdd8c7-7ad1-42d9-aa08-882334260d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83ea779112a4f9d8afbfe3fcec0ea08, user_id=5696d1c26bdf4324be3ff56176991860, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:07.792705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdd8c7-7ad1-42d9-aa08-882334260d5d admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:07.878654 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1318/2643] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:07 2022] PUT /identity/v3/projects/a83ea779112a4f9d8afbfe3fcec0ea08/users/5696d1c26bdf4324be3ff56176991860/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:07.882944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a5187-b750-439f-9a72-f8bb9c3ac3e0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:07.893033 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a5187-b750-439f-9a72-f8bb9c3ac3e0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:07.932468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:07.950524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed285d39-144f-4155-b8c9-c39603053123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:07.951417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed285d39-144f-4155-b8c9-c39603053123 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:07.951705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed285d39-144f-4155-b8c9-c39603053123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:07.952006 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed285d39-144f-4155-b8c9-c39603053123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:07.952695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed285d39-144f-4155-b8c9-c39603053123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:07.954315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed285d39-144f-4155-b8c9-c39603053123 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:07.970826 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1326/2644] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:30:07.975202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdd8c7-7ad1-42d9-aa08-882334260d5d admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:07.986435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdd8c7-7ad1-42d9-aa08-882334260d5d admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:07.991042 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4f241-2767-4087-ab63-e326c5cfccef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:07.991699 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f241-2767-4087-ab63-e326c5cfccef admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:07.992016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f241-2767-4087-ab63-e326c5cfccef admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:07.992304 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f241-2767-4087-ab63-e326c5cfccef admin admin] PATH_INFO: `/v3/projects/a83ea779112a4f9d8afbfe3fcec0ea08/users/5696d1c26bdf4324be3ff56176991860/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:07.994362 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4f241-2767-4087-ab63-e326c5cfccef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83ea779112a4f9d8afbfe3fcec0ea08, user_id=5696d1c26bdf4324be3ff56176991860, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:07.997078 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4f241-2767-4087-ab63-e326c5cfccef admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:08.007197 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1319/2645] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:07 2022] PUT /identity/v3/projects/a83ea779112a4f9d8afbfe3fcec0ea08/users/5696d1c26bdf4324be3ff56176991860/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:08.017411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533374b5-502e-40e6-ab57-056aafa7dcce None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:08.017705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533374b5-502e-40e6-ab57-056aafa7dcce None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:08.018015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533374b5-502e-40e6-ab57-056aafa7dcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:08.046508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-533374b5-502e-40e6-ab57-056aafa7dcce None None] MFA Rules not processed for user `5696d1c26bdf4324be3ff56176991860`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:08.047642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-533374b5-502e-40e6-ab57-056aafa7dcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:08.116149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1327/2646] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:08 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:08.126840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4f241-2767-4087-ab63-e326c5cfccef admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:08.128539 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ddd4f241-2767-4087-ab63-e326c5cfccef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:08.197138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4f241-2767-4087-ab63-e326c5cfccef admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:08.201121 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9e4db-8a93-4ff5-8d45-0b43c7fdfda6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:08.201786 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9e4db-8a93-4ff5-8d45-0b43c7fdfda6 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:08.202035 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9e4db-8a93-4ff5-8d45-0b43c7fdfda6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:08.202309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9e4db-8a93-4ff5-8d45-0b43c7fdfda6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:08.218288 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9e4db-8a93-4ff5-8d45-0b43c7fdfda6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:08.221254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9e4db-8a93-4ff5-8d45-0b43c7fdfda6 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:08.235542 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1320/2647] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:08 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:08.394629 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533374b5-502e-40e6-ab57-056aafa7dcce None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:08.463664 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533374b5-502e-40e6-ab57-056aafa7dcce None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:08.469606 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:08.470415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:08.470805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:08.472988 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] PATH_INFO: `/v3/users/d82edd60e65f45528f9f56d82797b9fb` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:08.476588 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d82edd60e65f45528f9f56d82797b9fb)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:08.478697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:08.747768 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd82edd60e65f45528f9f56d82797b9fb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:08.755142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd82edd60e65f45528f9f56d82797b9fb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:08.758033 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9e4db-8a93-4ff5-8d45-0b43c7fdfda6 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:08.759644 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b6c9e4db-8a93-4ff5-8d45-0b43c7fdfda6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:08.803005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:08.817490 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9e4db-8a93-4ff5-8d45-0b43c7fdfda6 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:08.821865 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab47ab-780c-4ed2-90e6-335ca549450a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:08.822516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab47ab-780c-4ed2-90e6-335ca549450a service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:08.822796 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab47ab-780c-4ed2-90e6-335ca549450a service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:08.823068 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab47ab-780c-4ed2-90e6-335ca549450a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:08.826791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-aaab47ab-780c-4ed2-90e6-335ca549450a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:08.838642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd82edd60e65f45528f9f56d82797b9fb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:08.839558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd82edd60e65f45528f9f56d82797b9fb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:08.839703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab47ab-780c-4ed2-90e6-335ca549450a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:08.843352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab47ab-780c-4ed2-90e6-335ca549450a service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:08.843453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:08.858534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd82edd60e65f45528f9f56d82797b9fb'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:08.859231 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1321/2648] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:08.865832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1328/2649] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:08 2022] DELETE /identity/v3/users/d82edd60e65f45528f9f56d82797b9fb => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:08.878075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78050718-2669-45b2-ba70-16a9389370b0 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:08.878075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78050718-2669-45b2-ba70-16a9389370b0 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:08.878075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78050718-2669-45b2-ba70-16a9389370b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:08.894323 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-78050718-2669-45b2-ba70-16a9389370b0 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:08.896658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-78050718-2669-45b2-ba70-16a9389370b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:08.962202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1322/2650] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:08.974098 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:08.975531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:09.049997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bda56d-0d53-45fe-8e03-c4c52c798609 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:09.054770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e37aa-3198-4c32-af49-754463b37dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:09.056198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e37aa-3198-4c32-af49-754463b37dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:09.056198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e37aa-3198-4c32-af49-754463b37dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:09.056198 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e37aa-3198-4c32-af49-754463b37dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:09.066836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e37aa-3198-4c32-af49-754463b37dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:09.068459 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e37aa-3198-4c32-af49-754463b37dd6 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:09.078279 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1329/2651] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:30:09.783464 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78050718-2669-45b2-ba70-16a9389370b0 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:09.785837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-78050718-2669-45b2-ba70-16a9389370b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:09.801225 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78050718-2669-45b2-ba70-16a9389370b0 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:09.807115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:09.807760 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:09.813868 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:09.813868 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] PATH_INFO: `/v3/projects/fb2e18e3fb414d12b9361c0dd9089d00` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:09.813868 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb2e18e3fb414d12b9361c0dd9089d00)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:09.813868 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:09.952347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb2e18e3fb414d12b9361c0dd9089d00', 'request_id': 'req-daefeba3-93e2-4c70-aaf7-5167af25583e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:09.960757 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb2e18e3fb414d12b9361c0dd9089d00', 'request_id': 'req-daefeba3-93e2-4c70-aaf7-5167af25583e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:10.068728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb2e18e3fb414d12b9361c0dd9089d00', 'request_id': 'req-daefeba3-93e2-4c70-aaf7-5167af25583e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:10.081393 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] The token cache is being invalidate because project fb2e18e3fb414d12b9361c0dd9089d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:10.081688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:10.082337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:10.083905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1323/2652] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:09 2022] DELETE /identity/v3/projects/fb2e18e3fb414d12b9361c0dd9089d00 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:10.087878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e37aa-3198-4c32-af49-754463b37dd6 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:10.098565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e37aa-3198-4c32-af49-754463b37dd6 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:10.156625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:10.157256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:10.157534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:10.157804 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] PATH_INFO: `/v3/users/5696d1c26bdf4324be3ff56176991860` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:10.159031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5696d1c26bdf4324be3ff56176991860)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:10.160964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:10.364841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5696d1c26bdf4324be3ff56176991860'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:10.375613 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5696d1c26bdf4324be3ff56176991860'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:10.446102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5696d1c26bdf4324be3ff56176991860'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:10.446803 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5696d1c26bdf4324be3ff56176991860'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:10.466634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e034abd3-d1d5-4064-a51c-1101ce948ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5696d1c26bdf4324be3ff56176991860'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:10.469894 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1330/2653] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:10 2022] DELETE /identity/v3/users/5696d1c26bdf4324be3ff56176991860 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:11.132943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:11.135357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:11.164082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daefeba3-93e2-4c70-aaf7-5167af25583e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:11.239282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:11.239905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:11.240196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:11.240477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] PATH_INFO: `/v3/projects/a83ea779112a4f9d8afbfe3fcec0ea08` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:11.241678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a83ea779112a4f9d8afbfe3fcec0ea08)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:11.244274 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:11.255357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:11.416559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a83ea779112a4f9d8afbfe3fcec0ea08', 'request_id': 'req-8243e83f-fbf3-4e43-ad0e-704e13845232'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:11.423635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a83ea779112a4f9d8afbfe3fcec0ea08', 'request_id': 'req-8243e83f-fbf3-4e43-ad0e-704e13845232'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:11.540750 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a83ea779112a4f9d8afbfe3fcec0ea08', 'request_id': 'req-8243e83f-fbf3-4e43-ad0e-704e13845232'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:11.554515 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] The token cache is being invalidate because project a83ea779112a4f9d8afbfe3fcec0ea08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:11.554866 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:11.555627 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8243e83f-fbf3-4e43-ad0e-704e13845232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:11.557440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1324/2654] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:11 2022] DELETE /identity/v3/projects/a83ea779112a4f9d8afbfe3fcec0ea08 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:11.562464 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4058f3a-c75b-48ad-a169-7fcda42b69f3 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:11.562949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4058f3a-c75b-48ad-a169-7fcda42b69f3 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:11.563167 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4058f3a-c75b-48ad-a169-7fcda42b69f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:11.571133 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:11.588251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-f4058f3a-c75b-48ad-a169-7fcda42b69f3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:11.589538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-f4058f3a-c75b-48ad-a169-7fcda42b69f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:11.650072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1331/2655] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:11.659968 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbaa26-7c37-4a92-af63-c3d734614fb2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:11.660259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbaa26-7c37-4a92-af63-c3d734614fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:11.660561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbaa26-7c37-4a92-af63-c3d734614fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:11.683532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-23dbaa26-7c37-4a92-af63-c3d734614fb2 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:11.683532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-23dbaa26-7c37-4a92-af63-c3d734614fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:11.689988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1325/2656] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:11.694250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4058f3a-c75b-48ad-a169-7fcda42b69f3 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:11.704726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4058f3a-c75b-48ad-a169-7fcda42b69f3 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:11.709561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0906121-bf11-43f4-aa89-e2f58b1a83a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:11.710176 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906121-bf11-43f4-aa89-e2f58b1a83a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:11.710456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906121-bf11-43f4-aa89-e2f58b1a83a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:11.714717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906121-bf11-43f4-aa89-e2f58b1a83a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:11.714717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0906121-bf11-43f4-aa89-e2f58b1a83a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:30:11.714717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0906121-bf11-43f4-aa89-e2f58b1a83a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:11.714717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0906121-bf11-43f4-aa89-e2f58b1a83a0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:11.725997 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1332/2657] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:30:11.730415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbaa26-7c37-4a92-af63-c3d734614fb2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:11.741646 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbaa26-7c37-4a92-af63-c3d734614fb2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:11.747005 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d489e03-fcd2-4198-8e4b-1bcf7bdb4b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:11.747731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d489e03-fcd2-4198-8e4b-1bcf7bdb4b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:11.748047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d489e03-fcd2-4198-8e4b-1bcf7bdb4b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:11.748361 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d489e03-fcd2-4198-8e4b-1bcf7bdb4b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:11.749027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d489e03-fcd2-4198-8e4b-1bcf7bdb4b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:11.750963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d489e03-fcd2-4198-8e4b-1bcf7bdb4b13 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:11.836423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1326/2658] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:11 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:11.841335 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0906121-bf11-43f4-aa89-e2f58b1a83a0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:11.851148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0906121-bf11-43f4-aa89-e2f58b1a83a0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:11.919083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad7a79-f2c6-417e-9b09-124a7c10e6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:11.919905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad7a79-f2c6-417e-9b09-124a7c10e6f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:11.920189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad7a79-f2c6-417e-9b09-124a7c10e6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:11.920465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad7a79-f2c6-417e-9b09-124a7c10e6f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:11.925725 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad7a79-f2c6-417e-9b09-124a7c10e6f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:11.927698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad7a79-f2c6-417e-9b09-124a7c10e6f0 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:12.047619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1333/2659] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:11 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:12.048991 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d489e03-fcd2-4198-8e4b-1bcf7bdb4b13 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:12.060834 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d489e03-fcd2-4198-8e4b-1bcf7bdb4b13 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:12.067144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993072f-c470-44ca-9fa8-514ec2752462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:12.067770 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993072f-c470-44ca-9fa8-514ec2752462 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:12.068128 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993072f-c470-44ca-9fa8-514ec2752462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:12.069315 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993072f-c470-44ca-9fa8-514ec2752462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:12.070037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993072f-c470-44ca-9fa8-514ec2752462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:12.071930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993072f-c470-44ca-9fa8-514ec2752462 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:12.092543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1327/2660] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:30:12.096448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad7a79-f2c6-417e-9b09-124a7c10e6f0 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:12.107189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad7a79-f2c6-417e-9b09-124a7c10e6f0 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:12.114781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efa2ae-118e-445d-905f-18a9cf3734f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:12.115582 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efa2ae-118e-445d-905f-18a9cf3734f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:12.115981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efa2ae-118e-445d-905f-18a9cf3734f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:12.116352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efa2ae-118e-445d-905f-18a9cf3734f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:12.117135 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31efa2ae-118e-445d-905f-18a9cf3734f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:12.118822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31efa2ae-118e-445d-905f-18a9cf3734f6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:12.135542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1334/2661] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:30:12.137326 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993072f-c470-44ca-9fa8-514ec2752462 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:12.149208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993072f-c470-44ca-9fa8-514ec2752462 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:12.157217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b94916-97b4-4765-9900-d3419992846c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:12.157908 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b94916-97b4-4765-9900-d3419992846c admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:12.158232 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b94916-97b4-4765-9900-d3419992846c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:12.163600 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b94916-97b4-4765-9900-d3419992846c admin admin] PATH_INFO: `/v3/projects/6ae7ad0660344d518186eb9bc1b74560/users/28040a4378694177acd3ccb00ee041b0/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:12.180158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b94916-97b4-4765-9900-d3419992846c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae7ad0660344d518186eb9bc1b74560, user_id=28040a4378694177acd3ccb00ee041b0, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:12.183641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b94916-97b4-4765-9900-d3419992846c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:12.292990 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1328/2662] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:12 2022] PUT /identity/v3/projects/6ae7ad0660344d518186eb9bc1b74560/users/28040a4378694177acd3ccb00ee041b0/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:12.303726 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601bc7a-24fe-4dd0-b6aa-0bcb1a7f7c89 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:12.304041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601bc7a-24fe-4dd0-b6aa-0bcb1a7f7c89 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:12.304318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601bc7a-24fe-4dd0-b6aa-0bcb1a7f7c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:12.343053 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-6601bc7a-24fe-4dd0-b6aa-0bcb1a7f7c89 None None] MFA Rules not processed for user `28040a4378694177acd3ccb00ee041b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:12.344299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6601bc7a-24fe-4dd0-b6aa-0bcb1a7f7c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:12.422022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:12.430465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1335/2663] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:12.438862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b94916-97b4-4765-9900-d3419992846c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:12.526842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b94916-97b4-4765-9900-d3419992846c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:12.532026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842ebd7-089c-4d70-b363-67b320db4c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:12.532650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842ebd7-089c-4d70-b363-67b320db4c86 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:12.532928 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842ebd7-089c-4d70-b363-67b320db4c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:12.533199 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842ebd7-089c-4d70-b363-67b320db4c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:12.533915 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842ebd7-089c-4d70-b363-67b320db4c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:12.536084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842ebd7-089c-4d70-b363-67b320db4c86 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:12.629525 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1329/2664] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:12 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:12.634779 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601bc7a-24fe-4dd0-b6aa-0bcb1a7f7c89 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:12.658509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601bc7a-24fe-4dd0-b6aa-0bcb1a7f7c89 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:12.743704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d4452-6564-4c5e-aa01-7558e004e067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:12.745049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4452-6564-4c5e-aa01-7558e004e067 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:12.745407 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4452-6564-4c5e-aa01-7558e004e067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:12.746872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d4452-6564-4c5e-aa01-7558e004e067 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:12.746872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d4452-6564-4c5e-aa01-7558e004e067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:12.748858 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d4452-6564-4c5e-aa01-7558e004e067 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:12.852737 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1336/2665] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:12 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:12.859311 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842ebd7-089c-4d70-b363-67b320db4c86 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:12.879234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842ebd7-089c-4d70-b363-67b320db4c86 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:12.887135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f944664-0a72-4fce-ba14-4bea973ae973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:12.888773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f944664-0a72-4fce-ba14-4bea973ae973 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:12.891303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f944664-0a72-4fce-ba14-4bea973ae973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:12.891634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f944664-0a72-4fce-ba14-4bea973ae973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:12.892514 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f944664-0a72-4fce-ba14-4bea973ae973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:12.894870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f944664-0a72-4fce-ba14-4bea973ae973 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:12.906506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1330/2666] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:12.910918 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d4452-6564-4c5e-aa01-7558e004e067 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:12.921824 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d4452-6564-4c5e-aa01-7558e004e067 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:12.926672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298738c-0023-48e5-9733-af3b33eaad25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:12.927360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298738c-0023-48e5-9733-af3b33eaad25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:12.927642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298738c-0023-48e5-9733-af3b33eaad25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:12.927972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298738c-0023-48e5-9733-af3b33eaad25 admin admin] PATH_INFO: `/v3/projects/65514f76ada04c87b8a136015a660483/users/332a7ebc8d054d78aa01a90087d347fb/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:12.945316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298738c-0023-48e5-9733-af3b33eaad25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65514f76ada04c87b8a136015a660483, user_id=332a7ebc8d054d78aa01a90087d347fb, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:12.948029 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298738c-0023-48e5-9733-af3b33eaad25 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:13.037965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1337/2667] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:12 2022] PUT /identity/v3/projects/65514f76ada04c87b8a136015a660483/users/332a7ebc8d054d78aa01a90087d347fb/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:13.041928 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f944664-0a72-4fce-ba14-4bea973ae973 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:13.052771 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f944664-0a72-4fce-ba14-4bea973ae973 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:13.056462 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:13.109426 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cc177-8c5e-4491-9357-67915eb76faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:13.109965 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cc177-8c5e-4491-9357-67915eb76faa admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:13.110250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cc177-8c5e-4491-9357-67915eb76faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:13.110519 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cc177-8c5e-4491-9357-67915eb76faa admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:13.111202 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cc177-8c5e-4491-9357-67915eb76faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:13.112825 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cc177-8c5e-4491-9357-67915eb76faa admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:13.123722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1331/2668] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:13.127985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298738c-0023-48e5-9733-af3b33eaad25 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:13.159164 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298738c-0023-48e5-9733-af3b33eaad25 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:13.164330 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182de1b-4fb1-4a54-a4f1-c7bcdadda0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:13.165299 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182de1b-4fb1-4a54-a4f1-c7bcdadda0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:13.170641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182de1b-4fb1-4a54-a4f1-c7bcdadda0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:13.170641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182de1b-4fb1-4a54-a4f1-c7bcdadda0d8 admin admin] PATH_INFO: `/v3/projects/65514f76ada04c87b8a136015a660483/users/332a7ebc8d054d78aa01a90087d347fb/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:13.170641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182de1b-4fb1-4a54-a4f1-c7bcdadda0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65514f76ada04c87b8a136015a660483, user_id=332a7ebc8d054d78aa01a90087d347fb, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:13.171945 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182de1b-4fb1-4a54-a4f1-c7bcdadda0d8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:13.187297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1338/2669] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:13 2022] PUT /identity/v3/projects/65514f76ada04c87b8a136015a660483/users/332a7ebc8d054d78aa01a90087d347fb/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:13.197592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ed232-f68c-4e6a-99f4-a6e2b5823afe None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:13.198115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ed232-f68c-4e6a-99f4-a6e2b5823afe None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:13.198218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ed232-f68c-4e6a-99f4-a6e2b5823afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:13.238718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5b5ed232-f68c-4e6a-99f4-a6e2b5823afe None None] MFA Rules not processed for user `332a7ebc8d054d78aa01a90087d347fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:13.240069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5b5ed232-f68c-4e6a-99f4-a6e2b5823afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:13.318741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1332/2670] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:13.330866 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182de1b-4fb1-4a54-a4f1-c7bcdadda0d8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:13.332682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-1182de1b-4fb1-4a54-a4f1-c7bcdadda0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:13.408342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182de1b-4fb1-4a54-a4f1-c7bcdadda0d8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:13.412367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9ba6f-1327-464e-89cc-4b0e01cca203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:13.412993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ba6f-1327-464e-89cc-4b0e01cca203 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:13.413273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ba6f-1327-464e-89cc-4b0e01cca203 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:13.413553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ba6f-1327-464e-89cc-4b0e01cca203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:13.424027 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9ba6f-1327-464e-89cc-4b0e01cca203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:13.425682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9ba6f-1327-464e-89cc-4b0e01cca203 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:13.440873 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1339/2671] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:14.526209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ed232-f68c-4e6a-99f4-a6e2b5823afe None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:14.608788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ed232-f68c-4e6a-99f4-a6e2b5823afe None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:14.614248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:14.614898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:14.615208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:14.615573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] PATH_INFO: `/v3/users/28040a4378694177acd3ccb00ee041b0` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:14.616877 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28040a4378694177acd3ccb00ee041b0)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:14.618841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:14.781645 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:14.797499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28040a4378694177acd3ccb00ee041b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:14.805543 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28040a4378694177acd3ccb00ee041b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:14.886263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28040a4378694177acd3ccb00ee041b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:14.886979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28040a4378694177acd3ccb00ee041b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:14.887883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28040a4378694177acd3ccb00ee041b0'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:14.908147 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1333/2672] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:14 2022] DELETE /identity/v3/users/28040a4378694177acd3ccb00ee041b0 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:30:14.912825 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd63d99-de13-4e5c-a4ef-3c140f99286b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:14.913356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd63d99-de13-4e5c-a4ef-3c140f99286b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:14.913356 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd63d99-de13-4e5c-a4ef-3c140f99286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:14.933057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-bfd63d99-de13-4e5c-a4ef-3c140f99286b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:14.934295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bfd63d99-de13-4e5c-a4ef-3c140f99286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:14.983367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:14.999868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1340/2673] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:15.007608 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:15.009728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:15.090409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5856c-4531-4ca0-8801-41d8dc0bed66 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:15.098488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42da639-8ee8-40e1-9eda-19a20dec5b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:15.099407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da639-8ee8-40e1-9eda-19a20dec5b52 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:15.099857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da639-8ee8-40e1-9eda-19a20dec5b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:15.100772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da639-8ee8-40e1-9eda-19a20dec5b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:15.129718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42da639-8ee8-40e1-9eda-19a20dec5b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:15.131435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42da639-8ee8-40e1-9eda-19a20dec5b52 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:15.143997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1334/2674] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:15.678297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd63d99-de13-4e5c-a4ef-3c140f99286b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:15.679800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bfd63d99-de13-4e5c-a4ef-3c140f99286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:15.706397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd63d99-de13-4e5c-a4ef-3c140f99286b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:15.720637 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:15.721633 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:15.722066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:15.722436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] PATH_INFO: `/v3/projects/6ae7ad0660344d518186eb9bc1b74560` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:15.726344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae7ad0660344d518186eb9bc1b74560)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:15.729667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:15.877111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae7ad0660344d518186eb9bc1b74560', 'request_id': 'req-05eae831-b376-41c6-8c92-a5f1f78b6499'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:15.884153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae7ad0660344d518186eb9bc1b74560', 'request_id': 'req-05eae831-b376-41c6-8c92-a5f1f78b6499'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:15.892224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae7ad0660344d518186eb9bc1b74560', 'request_id': 'req-05eae831-b376-41c6-8c92-a5f1f78b6499'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:15.980355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] The token cache is being invalidate because project 6ae7ad0660344d518186eb9bc1b74560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:15.980955 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:15.981311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:15.983363 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1341/2675] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:15 2022] DELETE /identity/v3/projects/6ae7ad0660344d518186eb9bc1b74560 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:15.987818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42da639-8ee8-40e1-9eda-19a20dec5b52 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:15.997784 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42da639-8ee8-40e1-9eda-19a20dec5b52 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:16.072617 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:16.074773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:16.074773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:16.074773 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] PATH_INFO: `/v3/users/332a7ebc8d054d78aa01a90087d347fb` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:16.077553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] RBAC: Authorizing `identity:delete_user(user_id=332a7ebc8d054d78aa01a90087d347fb)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:16.082537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:16.279898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '332a7ebc8d054d78aa01a90087d347fb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:16.284937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:16.287870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '332a7ebc8d054d78aa01a90087d347fb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:16.406752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '332a7ebc8d054d78aa01a90087d347fb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:16.407475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '332a7ebc8d054d78aa01a90087d347fb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:16.408342 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '332a7ebc8d054d78aa01a90087d347fb'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:16.428606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1335/2676] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:15 2022] DELETE /identity/v3/users/332a7ebc8d054d78aa01a90087d347fb => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:16.908096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:16.908599 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:16.938426 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eae831-b376-41c6-8c92-a5f1f78b6499 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:17.017849 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:17.018530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.018790 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.019066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] PATH_INFO: `/v3/projects/65514f76ada04c87b8a136015a660483` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.020287 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=65514f76ada04c87b8a136015a660483)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:17.022378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:17.123265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:17.125930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:17.141017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bc6e9-2353-40ab-99d7-b037c55c310a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:17.145910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65514f76ada04c87b8a136015a660483', 'request_id': 'req-2241a2d6-5b73-4d63-9bac-143990a510fc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:17.152148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65514f76ada04c87b8a136015a660483', 'request_id': 'req-2241a2d6-5b73-4d63-9bac-143990a510fc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:17.158912 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65514f76ada04c87b8a136015a660483', 'request_id': 'req-2241a2d6-5b73-4d63-9bac-143990a510fc'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:17.200173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:17.200827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.201112 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.201395 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] PATH_INFO: `/v3/users/aa886e3c6aac40c8975bad310f65e517` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.202579 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa886e3c6aac40c8975bad310f65e517)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:17.204622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:17.235470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] The token cache is being invalidate because project 65514f76ada04c87b8a136015a660483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:17.235762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:17.236429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-2241a2d6-5b73-4d63-9bac-143990a510fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:17.237935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1342/2677] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:16 2022] DELETE /identity/v3/projects/65514f76ada04c87b8a136015a660483 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:17.242560 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f265e2c-2ebb-41c0-aea9-44e7d0d9c270 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.242845 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f265e2c-2ebb-41c0-aea9-44e7d0d9c270 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.243124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f265e2c-2ebb-41c0-aea9-44e7d0d9c270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.257161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:17.262662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-0f265e2c-2ebb-41c0-aea9-44e7d0d9c270 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:17.267495 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0f265e2c-2ebb-41c0-aea9-44e7d0d9c270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:17.271914 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1343/2678] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:30:17.281100 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93a44b-0081-469d-9c7c-36821d7c639f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.281382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93a44b-0081-469d-9c7c-36821d7c639f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.281654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93a44b-0081-469d-9c7c-36821d7c639f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.300889 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5b93a44b-0081-469d-9c7c-36821d7c639f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:17.302065 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5b93a44b-0081-469d-9c7c-36821d7c639f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:17.310349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1344/2679] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:30:17.314384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93a44b-0081-469d-9c7c-36821d7c639f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:17.323436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93a44b-0081-469d-9c7c-36821d7c639f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:17.328691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f0037-6652-41ed-be70-2459ceb1839b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:17.329995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f0037-6652-41ed-be70-2459ceb1839b admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.330280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f0037-6652-41ed-be70-2459ceb1839b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.330558 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f0037-6652-41ed-be70-2459ceb1839b admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.331282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f0037-6652-41ed-be70-2459ceb1839b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:30:17.331591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f0037-6652-41ed-be70-2459ceb1839b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:17.333273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f0037-6652-41ed-be70-2459ceb1839b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:17.345231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1345/2680] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 01:30:17.349496 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f0037-6652-41ed-be70-2459ceb1839b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:17.358759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f0037-6652-41ed-be70-2459ceb1839b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:17.363532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70f143-f8ed-4db9-9473-875eff5cd1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:17.364282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70f143-f8ed-4db9-9473-875eff5cd1db admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.364509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70f143-f8ed-4db9-9473-875eff5cd1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.364789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70f143-f8ed-4db9-9473-875eff5cd1db admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.365473 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70f143-f8ed-4db9-9473-875eff5cd1db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:17.367367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70f143-f8ed-4db9-9473-875eff5cd1db admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:17.548658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa886e3c6aac40c8975bad310f65e517'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:17.555469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa886e3c6aac40c8975bad310f65e517'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:17.596655 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1346/2681] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:17 2022] POST /identity/v3/projects => generated 365 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:30:17.600958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70f143-f8ed-4db9-9473-875eff5cd1db admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:17.661061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70f143-f8ed-4db9-9473-875eff5cd1db admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:17.666227 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa886e3c6aac40c8975bad310f65e517'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:17.666927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa886e3c6aac40c8975bad310f65e517'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:17.667282 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9dabb-9681-4b02-bd88-2099b0589662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:17.667559 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa886e3c6aac40c8975bad310f65e517'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:17.668015 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9dabb-9681-4b02-bd88-2099b0589662 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.668305 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9dabb-9681-4b02-bd88-2099b0589662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.668580 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9dabb-9681-4b02-bd88-2099b0589662 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.669253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9dabb-9681-4b02-bd88-2099b0589662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:17.671288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9dabb-9681-4b02-bd88-2099b0589662 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:17.687074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1336/2682] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:17 2022] DELETE /identity/v3/users/aa886e3c6aac40c8975bad310f65e517 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:30:17.753644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1347/2683] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:17 2022] POST /identity/v3/users => generated 283 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:30:17.759356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:17.760901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:17.817891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:17.833476 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ec3e-2b31-45a6-80b8-8cffcacd624b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:17.838257 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf240110-ff3c-4924-a991-58638f24ebe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:17.840658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf240110-ff3c-4924-a991-58638f24ebe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.840658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf240110-ff3c-4924-a991-58638f24ebe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.840658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf240110-ff3c-4924-a991-58638f24ebe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.840658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf240110-ff3c-4924-a991-58638f24ebe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:17.841842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf240110-ff3c-4924-a991-58638f24ebe1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:17.852229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1337/2684] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:17.856303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9dabb-9681-4b02-bd88-2099b0589662 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:17.866692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9dabb-9681-4b02-bd88-2099b0589662 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:17.871942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1799c-73f2-4d2e-9200-f9365ad5e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:17.872550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1799c-73f2-4d2e-9200-f9365ad5e0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.872832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1799c-73f2-4d2e-9200-f9365ad5e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.873108 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1799c-73f2-4d2e-9200-f9365ad5e0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.873805 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1799c-73f2-4d2e-9200-f9365ad5e0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:17.875375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1799c-73f2-4d2e-9200-f9365ad5e0e8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:17.885203 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1348/2685] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:17.889929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf240110-ff3c-4924-a991-58638f24ebe1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:17.899763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf240110-ff3c-4924-a991-58638f24ebe1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:17.904435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c910a-1bac-4aa1-834e-356ceb78ca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:17.905123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c910a-1bac-4aa1-834e-356ceb78ca80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.905409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c910a-1bac-4aa1-834e-356ceb78ca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.905689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c910a-1bac-4aa1-834e-356ceb78ca80 admin admin] PATH_INFO: `/v3/projects/47906e660c1b45778be914c8eac97039/users/2212d011643c4f4580e550173a8b163c/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.922508 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c910a-1bac-4aa1-834e-356ceb78ca80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47906e660c1b45778be914c8eac97039, user_id=2212d011643c4f4580e550173a8b163c, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:17.925537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c910a-1bac-4aa1-834e-356ceb78ca80 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:17.953962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1799c-73f2-4d2e-9200-f9365ad5e0e8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:17.955918 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d0e1799c-73f2-4d2e-9200-f9365ad5e0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:17.980064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1799c-73f2-4d2e-9200-f9365ad5e0e8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:17.987630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:17.988670 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:17.989077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:17.989491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] PATH_INFO: `/v3/projects/1625d1429d294a2193fb22b2104384bf` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:17.990855 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1625d1429d294a2193fb22b2104384bf)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:17.994063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:18.029235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1338/2686] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:17 2022] PUT /identity/v3/projects/47906e660c1b45778be914c8eac97039/users/2212d011643c4f4580e550173a8b163c/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:18.039561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a25613-edd5-4d4c-891e-78fa7b83e9cc None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:18.040123 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a25613-edd5-4d4c-891e-78fa7b83e9cc None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:18.041530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a25613-edd5-4d4c-891e-78fa7b83e9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:18.072675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-22a25613-edd5-4d4c-891e-78fa7b83e9cc None None] MFA Rules not processed for user `2212d011643c4f4580e550173a8b163c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:18.074277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-22a25613-edd5-4d4c-891e-78fa7b83e9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:18.093554 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1625d1429d294a2193fb22b2104384bf', 'request_id': 'req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:18.101178 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1625d1429d294a2193fb22b2104384bf', 'request_id': 'req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:18.108454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1625d1429d294a2193fb22b2104384bf', 'request_id': 'req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:18.153624 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1339/2687] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:18.158181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a25613-edd5-4d4c-891e-78fa7b83e9cc None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:18.192634 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] The token cache is being invalidate because project 1625d1429d294a2193fb22b2104384bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:18.209281 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:18.214947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:18.216531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1349/2688] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:17 2022] DELETE /identity/v3/projects/1625d1429d294a2193fb22b2104384bf => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:30:18.221291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:18.222771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:18.249390 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a25613-edd5-4d4c-891e-78fa7b83e9cc None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:18.340034 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28c9d9-3425-4ed3-ab6f-8826a5faf1d9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:18.343095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c7fc7-1a1d-4bb7-b338-fd9e6315ab63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:18.343840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c7fc7-1a1d-4bb7-b338-fd9e6315ab63 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:18.344262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c7fc7-1a1d-4bb7-b338-fd9e6315ab63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:18.344555 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c7fc7-1a1d-4bb7-b338-fd9e6315ab63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:18.344961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:18.345409 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c7fc7-1a1d-4bb7-b338-fd9e6315ab63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:18.345773 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:18.347427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:18.347724 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c7fc7-1a1d-4bb7-b338-fd9e6315ab63 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:18.347879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] PATH_INFO: `/v3/users/d327cdb7279d487289c113b25bf59426` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:18.356961 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d327cdb7279d487289c113b25bf59426)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:18.359608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:18.441268 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1340/2689] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:18 2022] POST /identity/v3/projects => generated 365 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:30:18.445411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c7fc7-1a1d-4bb7-b338-fd9e6315ab63 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:18.447209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-0d3c7fc7-1a1d-4bb7-b338-fd9e6315ab63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:18.523595 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c7fc7-1a1d-4bb7-b338-fd9e6315ab63 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:18.529172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c696df-e2ff-407d-b97d-899e23338d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:18.529951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c696df-e2ff-407d-b97d-899e23338d9a admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:18.530352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c696df-e2ff-407d-b97d-899e23338d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:18.530633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c696df-e2ff-407d-b97d-899e23338d9a admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:18.531402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c696df-e2ff-407d-b97d-899e23338d9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:18.533488 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c696df-e2ff-407d-b97d-899e23338d9a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:18.545657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd327cdb7279d487289c113b25bf59426'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:18.552707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd327cdb7279d487289c113b25bf59426'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:18.643144 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1341/2690] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:18 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:30:18.647058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c696df-e2ff-407d-b97d-899e23338d9a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:18.657212 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c696df-e2ff-407d-b97d-899e23338d9a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:18.663517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd327cdb7279d487289c113b25bf59426'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:18.663517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:18.663517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd327cdb7279d487289c113b25bf59426'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:18.663880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db098d2-e2a6-4e55-ae2c-377b81b1d8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:18.663880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db098d2-e2a6-4e55-ae2c-377b81b1d8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:18.664384 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db098d2-e2a6-4e55-ae2c-377b81b1d8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:18.664666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db098d2-e2a6-4e55-ae2c-377b81b1d8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:18.665351 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db098d2-e2a6-4e55-ae2c-377b81b1d8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:18.666951 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db098d2-e2a6-4e55-ae2c-377b81b1d8f4 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:18.677947 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd327cdb7279d487289c113b25bf59426'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:18.682656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1342/2691] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:30:18.683816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db098d2-e2a6-4e55-ae2c-377b81b1d8f4 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:18.685723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4db098d2-e2a6-4e55-ae2c-377b81b1d8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:18.687528 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1350/2692] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:18 2022] DELETE /identity/v3/users/d327cdb7279d487289c113b25bf59426 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:30:18.698346 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db098d2-e2a6-4e55-ae2c-377b81b1d8f4 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:18.757639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eac7ce-07f5-46a0-955e-bff7afbeb300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:18.758317 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eac7ce-07f5-46a0-955e-bff7afbeb300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:18.758600 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eac7ce-07f5-46a0-955e-bff7afbeb300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:18.758869 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eac7ce-07f5-46a0-955e-bff7afbeb300 admin admin] PATH_INFO: `/v3/projects/706411e8e50e433a93553b6db165fe0e/users/0c587d22b9374e02a4e8dc91e47e17a7/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:18.768404 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:18.775894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eac7ce-07f5-46a0-955e-bff7afbeb300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=706411e8e50e433a93553b6db165fe0e, user_id=0c587d22b9374e02a4e8dc91e47e17a7, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:18.778369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eac7ce-07f5-46a0-955e-bff7afbeb300 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:18.863722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1343/2693] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:18 2022] PUT /identity/v3/projects/706411e8e50e433a93553b6db165fe0e/users/0c587d22b9374e02a4e8dc91e47e17a7/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:30:18.867675 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:18.877775 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5edb48-9663-4017-9909-e75514174f3e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:18.933267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1a867-7153-4fa6-91d6-f601aa093bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:18.933829 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1a867-7153-4fa6-91d6-f601aa093bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:18.934111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1a867-7153-4fa6-91d6-f601aa093bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:18.934433 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1a867-7153-4fa6-91d6-f601aa093bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:18.935102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1a867-7153-4fa6-91d6-f601aa093bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:18.936753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a1a867-7153-4fa6-91d6-f601aa093bd8 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:18.946132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1351/2694] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:18.951234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eac7ce-07f5-46a0-955e-bff7afbeb300 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:18.962244 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1a867-7153-4fa6-91d6-f601aa093bd8 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:18.966157 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-42a1a867-7153-4fa6-91d6-f601aa093bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:18.972048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eac7ce-07f5-46a0-955e-bff7afbeb300 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:18.977639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6db7a-680d-4f78-b292-0bddc4808598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:18.978326 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6db7a-680d-4f78-b292-0bddc4808598 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:18.978607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6db7a-680d-4f78-b292-0bddc4808598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:18.978881 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6db7a-680d-4f78-b292-0bddc4808598 admin admin] PATH_INFO: `/v3/projects/706411e8e50e433a93553b6db165fe0e/users/0c587d22b9374e02a4e8dc91e47e17a7/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:18.981054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6db7a-680d-4f78-b292-0bddc4808598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=706411e8e50e433a93553b6db165fe0e, user_id=0c587d22b9374e02a4e8dc91e47e17a7, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:18.983704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6db7a-680d-4f78-b292-0bddc4808598 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:18.993946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1a867-7153-4fa6-91d6-f601aa093bd8 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:19.012209 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1344/2695] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:18 2022] PUT /identity/v3/projects/706411e8e50e433a93553b6db165fe0e/users/0c587d22b9374e02a4e8dc91e47e17a7/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:19.020900 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73837e76-c95c-4e5b-8df2-ed9126c8cd34 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:19.021195 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73837e76-c95c-4e5b-8df2-ed9126c8cd34 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:19.021523 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73837e76-c95c-4e5b-8df2-ed9126c8cd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:19.053383 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-73837e76-c95c-4e5b-8df2-ed9126c8cd34 None None] MFA Rules not processed for user `0c587d22b9374e02a4e8dc91e47e17a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:19.054778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-73837e76-c95c-4e5b-8df2-ed9126c8cd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:19.066802 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:19.067457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:19.067741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:19.068052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] PATH_INFO: `/v3/projects/dc146b11964e4d6f9833033f987ff310` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:19.069324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc146b11964e4d6f9833033f987ff310)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:19.071253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:19.136299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1345/2696] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:30:19.148320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73837e76-c95c-4e5b-8df2-ed9126c8cd34 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:19.149755 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-73837e76-c95c-4e5b-8df2-ed9126c8cd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:19.178514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc146b11964e4d6f9833033f987ff310', 'request_id': 'req-bdf296a8-22f4-4019-945f-1166ca13470b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:19.187318 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc146b11964e4d6f9833033f987ff310', 'request_id': 'req-bdf296a8-22f4-4019-945f-1166ca13470b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:19.193168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc146b11964e4d6f9833033f987ff310', 'request_id': 'req-bdf296a8-22f4-4019-945f-1166ca13470b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:19.208486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73837e76-c95c-4e5b-8df2-ed9126c8cd34 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:19.256192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c5b0c-25b4-4827-aa47-015772bf1ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:19.256815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c5b0c-25b4-4827-aa47-015772bf1ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:19.257094 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c5b0c-25b4-4827-aa47-015772bf1ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:19.257365 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c5b0c-25b4-4827-aa47-015772bf1ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:19.276851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] The token cache is being invalidate because project dc146b11964e4d6f9833033f987ff310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:19.277174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:19.277807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:19.279331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1352/2697] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:18 2022] DELETE /identity/v3/projects/dc146b11964e4d6f9833033f987ff310 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:30:19.327158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c5b0c-25b4-4827-aa47-015772bf1ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:19.328684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c5b0c-25b4-4827-aa47-015772bf1ea8 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:19.339283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1346/2698] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:30:19.520988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:19.563919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:19.565660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:19.580129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf296a8-22f4-4019-945f-1166ca13470b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:19.585595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346ea8d-2e97-417b-9034-5f1ec36c6405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:19.586508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346ea8d-2e97-417b-9034-5f1ec36c6405 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:19.586958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346ea8d-2e97-417b-9034-5f1ec36c6405 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:19.587618 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346ea8d-2e97-417b-9034-5f1ec36c6405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:19.590206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b346ea8d-2e97-417b-9034-5f1ec36c6405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:19.598534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:19.662311 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346ea8d-2e97-417b-9034-5f1ec36c6405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:19.663755 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346ea8d-2e97-417b-9034-5f1ec36c6405 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:19.673978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1353/2699] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:30:29.757663 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=90 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:29.935231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:31.580451 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c5b0c-25b4-4827-aa47-015772bf1ea8 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:31.682913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c5b0c-25b4-4827-aa47-015772bf1ea8 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:31.687577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:31.688259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:31.688765 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:31.688884 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] PATH_INFO: `/v3/users/2212d011643c4f4580e550173a8b163c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:31.690001 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2212d011643c4f4580e550173a8b163c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:31.692577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:31.876541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2212d011643c4f4580e550173a8b163c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:31.883473 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2212d011643c4f4580e550173a8b163c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:31.948307 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2212d011643c4f4580e550173a8b163c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:31.948921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2212d011643c4f4580e550173a8b163c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:31.949526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2212d011643c4f4580e550173a8b163c'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:31.967430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1347/2700] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:31 2022] DELETE /identity/v3/users/2212d011643c4f4580e550173a8b163c => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:31.972018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516560-6dad-4b43-ad70-a52bfe9fc57b None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:31.972484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516560-6dad-4b43-ad70-a52bfe9fc57b None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:31.972625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516560-6dad-4b43-ad70-a52bfe9fc57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:32.021092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7b516560-6dad-4b43-ad70-a52bfe9fc57b None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:32.022280 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7b516560-6dad-4b43-ad70-a52bfe9fc57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:32.082349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1354/2701] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:32.092316 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:32.093694 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:32.134860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:32.166084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27453ab3-041d-42b9-9809-0c858bf54b96 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:32.169751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9cba2-8f62-4e09-915d-204485957cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:32.170371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9cba2-8f62-4e09-915d-204485957cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:32.170653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9cba2-8f62-4e09-915d-204485957cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:32.170933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9cba2-8f62-4e09-915d-204485957cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:32.180066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9cba2-8f62-4e09-915d-204485957cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:32.181503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9cba2-8f62-4e09-915d-204485957cd1 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:32.191238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1348/2702] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:32.584387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516560-6dad-4b43-ad70-a52bfe9fc57b None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:32.585994 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7b516560-6dad-4b43-ad70-a52bfe9fc57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:32.598033 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516560-6dad-4b43-ad70-a52bfe9fc57b None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:32.602642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:32.603291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:32.603573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:32.603900 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] PATH_INFO: `/v3/projects/47906e660c1b45778be914c8eac97039` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:32.605068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47906e660c1b45778be914c8eac97039)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:32.606964 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:32.713776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47906e660c1b45778be914c8eac97039', 'request_id': 'req-4d905c47-73db-41bc-a140-1109ca93a957'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:32.720362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47906e660c1b45778be914c8eac97039', 'request_id': 'req-4d905c47-73db-41bc-a140-1109ca93a957'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:32.725763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47906e660c1b45778be914c8eac97039', 'request_id': 'req-4d905c47-73db-41bc-a140-1109ca93a957'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:32.807801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] The token cache is being invalidate because project 47906e660c1b45778be914c8eac97039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:32.808124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:32.808760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:32.810285 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1355/2703] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:32 2022] DELETE /identity/v3/projects/47906e660c1b45778be914c8eac97039 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:32.814308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9cba2-8f62-4e09-915d-204485957cd1 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:32.823966 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9cba2-8f62-4e09-915d-204485957cd1 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:32.881172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:32.881813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:32.882097 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:32.882377 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] PATH_INFO: `/v3/users/0c587d22b9374e02a4e8dc91e47e17a7` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:32.883573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c587d22b9374e02a4e8dc91e47e17a7)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:32.885624 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:33.044388 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c587d22b9374e02a4e8dc91e47e17a7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:33.051170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c587d22b9374e02a4e8dc91e47e17a7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:33.114007 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c587d22b9374e02a4e8dc91e47e17a7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:33.114646 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c587d22b9374e02a4e8dc91e47e17a7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:33.115254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f7f1ce8b-6c0b-4e26-bb67-a816e2b91820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c587d22b9374e02a4e8dc91e47e17a7'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:33.132718 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1349/2704] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:32 2022] DELETE /identity/v3/users/0c587d22b9374e02a4e8dc91e47e17a7 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:33.524788 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:33.526205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:33.539186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d905c47-73db-41bc-a140-1109ca93a957 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:33.593160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:33.599414 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:33.600094 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:33.600379 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:33.600659 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] PATH_INFO: `/v3/projects/706411e8e50e433a93553b6db165fe0e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:33.601813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=706411e8e50e433a93553b6db165fe0e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:33.603711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:33.718959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '706411e8e50e433a93553b6db165fe0e', 'request_id': 'req-98e3092e-4898-4b2c-b38a-024321f35cd2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:33.725105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '706411e8e50e433a93553b6db165fe0e', 'request_id': 'req-98e3092e-4898-4b2c-b38a-024321f35cd2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:33.730511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '706411e8e50e433a93553b6db165fe0e', 'request_id': 'req-98e3092e-4898-4b2c-b38a-024321f35cd2'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:33.811377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] The token cache is being invalidate because project 706411e8e50e433a93553b6db165fe0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:33.811679 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:33.812491 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-98e3092e-4898-4b2c-b38a-024321f35cd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:33.814095 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1356/2705] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:33 2022] DELETE /identity/v3/projects/706411e8e50e433a93553b6db165fe0e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:33.819203 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0e62f-1f2b-405c-9d57-b132eee88043 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:33.819667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0e62f-1f2b-405c-9d57-b132eee88043 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:33.819810 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0e62f-1f2b-405c-9d57-b132eee88043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:33.868465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3fd0e62f-1f2b-405c-9d57-b132eee88043 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:33.869657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3fd0e62f-1f2b-405c-9d57-b132eee88043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:33.882684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:33.927399 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1350/2706] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:33.935114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f050ee1-3cca-4590-ad46-8a601aa68bba None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:33.935506 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f050ee1-3cca-4590-ad46-8a601aa68bba None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:33.936339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f050ee1-3cca-4590-ad46-8a601aa68bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:33.955601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-7f050ee1-3cca-4590-ad46-8a601aa68bba None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:33.956836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7f050ee1-3cca-4590-ad46-8a601aa68bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:33.965083 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1357/2707] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:33.969289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0e62f-1f2b-405c-9d57-b132eee88043 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:33.982253 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0e62f-1f2b-405c-9d57-b132eee88043 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:33.986136 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5ac28-6261-454f-a91f-1ea807f8be94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:33.986749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5ac28-6261-454f-a91f-1ea807f8be94 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:33.987030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5ac28-6261-454f-a91f-1ea807f8be94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:33.987306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5ac28-6261-454f-a91f-1ea807f8be94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:33.988034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5ac28-6261-454f-a91f-1ea807f8be94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:30:33.988341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5ac28-6261-454f-a91f-1ea807f8be94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:33.989945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5ac28-6261-454f-a91f-1ea807f8be94 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:34.001813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1351/2708] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:30:34.006125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f050ee1-3cca-4590-ad46-8a601aa68bba None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:34.015387 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f050ee1-3cca-4590-ad46-8a601aa68bba None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:34.020087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e142d-ce7b-4b50-af1f-babbc2c34479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:34.020694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e142d-ce7b-4b50-af1f-babbc2c34479 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:34.020977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e142d-ce7b-4b50-af1f-babbc2c34479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:34.021254 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e142d-ce7b-4b50-af1f-babbc2c34479 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:34.021913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e142d-ce7b-4b50-af1f-babbc2c34479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:34.023799 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e142d-ce7b-4b50-af1f-babbc2c34479 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:34.179128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1358/2709] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:34 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:34.183229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5ac28-6261-454f-a91f-1ea807f8be94 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:34.193121 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5ac28-6261-454f-a91f-1ea807f8be94 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:34.248129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf43ff6-5e17-4591-bae7-8d4a7ba882d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:34.248775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf43ff6-5e17-4591-bae7-8d4a7ba882d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:34.249060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf43ff6-5e17-4591-bae7-8d4a7ba882d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:34.249335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf43ff6-5e17-4591-bae7-8d4a7ba882d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:34.249999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf43ff6-5e17-4591-bae7-8d4a7ba882d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:34.251952 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf43ff6-5e17-4591-bae7-8d4a7ba882d7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:34.349720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1352/2710] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:34 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:34.353906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e142d-ce7b-4b50-af1f-babbc2c34479 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:34.365340 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e142d-ce7b-4b50-af1f-babbc2c34479 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:34.369960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf8fe4-f5a7-4460-a500-f55cd393b7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:34.370597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf8fe4-f5a7-4460-a500-f55cd393b7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:34.370917 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf8fe4-f5a7-4460-a500-f55cd393b7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:34.371195 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf8fe4-f5a7-4460-a500-f55cd393b7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:34.371919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf8fe4-f5a7-4460-a500-f55cd393b7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:34.373564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf8fe4-f5a7-4460-a500-f55cd393b7a9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:34.383774 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1359/2711] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:34.387998 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf43ff6-5e17-4591-bae7-8d4a7ba882d7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:34.397955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf43ff6-5e17-4591-bae7-8d4a7ba882d7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:34.402534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048eba48-1eb7-4c4d-bf34-9bc6ddbf5792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:34.403152 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048eba48-1eb7-4c4d-bf34-9bc6ddbf5792 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:34.403433 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048eba48-1eb7-4c4d-bf34-9bc6ddbf5792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:34.403708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048eba48-1eb7-4c4d-bf34-9bc6ddbf5792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:34.404947 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048eba48-1eb7-4c4d-bf34-9bc6ddbf5792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:34.406537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048eba48-1eb7-4c4d-bf34-9bc6ddbf5792 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:34.416860 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1353/2712] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:34.421601 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf8fe4-f5a7-4460-a500-f55cd393b7a9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:34.431425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf8fe4-f5a7-4460-a500-f55cd393b7a9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:34.435969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26ac3f-20e4-4ae7-b54b-2fa2793af89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:34.436619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26ac3f-20e4-4ae7-b54b-2fa2793af89b admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:34.436902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26ac3f-20e4-4ae7-b54b-2fa2793af89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:34.437189 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26ac3f-20e4-4ae7-b54b-2fa2793af89b admin admin] PATH_INFO: `/v3/projects/c12c2059199042bcaaaf1d4131483812/users/14e6abe7564047f7ac377229f01704ac/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:34.451185 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26ac3f-20e4-4ae7-b54b-2fa2793af89b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12c2059199042bcaaaf1d4131483812, user_id=14e6abe7564047f7ac377229f01704ac, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:34.453862 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26ac3f-20e4-4ae7-b54b-2fa2793af89b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:34.517083 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048eba48-1eb7-4c4d-bf34-9bc6ddbf5792 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:34.518890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-048eba48-1eb7-4c4d-bf34-9bc6ddbf5792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:34.526386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1360/2713] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:34 2022] PUT /identity/v3/projects/c12c2059199042bcaaaf1d4131483812/users/14e6abe7564047f7ac377229f01704ac/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:30:34.537720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad972c0b-5956-44de-a2ed-bc786a57dda7 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:34.538376 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad972c0b-5956-44de-a2ed-bc786a57dda7 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:34.538667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad972c0b-5956-44de-a2ed-bc786a57dda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:34.566046 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ad972c0b-5956-44de-a2ed-bc786a57dda7 None None] MFA Rules not processed for user `14e6abe7564047f7ac377229f01704ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:34.567410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ad972c0b-5956-44de-a2ed-bc786a57dda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:34.589910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048eba48-1eb7-4c4d-bf34-9bc6ddbf5792 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:34.635283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:34.640283 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1361/2714] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:30:34.645045 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad972c0b-5956-44de-a2ed-bc786a57dda7 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:34.645788 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b547fd8-cfce-4df5-a4b7-8eb5cbd951c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:34.646410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b547fd8-cfce-4df5-a4b7-8eb5cbd951c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:34.646688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b547fd8-cfce-4df5-a4b7-8eb5cbd951c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:34.647142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b547fd8-cfce-4df5-a4b7-8eb5cbd951c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:34.648993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8b547fd8-cfce-4df5-a4b7-8eb5cbd951c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:34.659051 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:34.709411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b547fd8-cfce-4df5-a4b7-8eb5cbd951c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:34.709888 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad972c0b-5956-44de-a2ed-bc786a57dda7 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:34.711050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b547fd8-cfce-4df5-a4b7-8eb5cbd951c8 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:34.714561 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e39bfd-ef71-4957-adc3-10889e66388a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:34.715179 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39bfd-ef71-4957-adc3-10889e66388a admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:34.715465 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39bfd-ef71-4957-adc3-10889e66388a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:34.715828 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e39bfd-ef71-4957-adc3-10889e66388a admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:34.716648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e39bfd-ef71-4957-adc3-10889e66388a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:34.719061 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e39bfd-ef71-4957-adc3-10889e66388a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:34.720601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1354/2715] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:34.819548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1362/2716] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:34 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:30:34.823707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b547fd8-cfce-4df5-a4b7-8eb5cbd951c8 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:34.834250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b547fd8-cfce-4df5-a4b7-8eb5cbd951c8 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:34.895997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e35679-72a4-4d13-9074-3321fed6cc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:34.896633 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e35679-72a4-4d13-9074-3321fed6cc1e admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:34.896918 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e35679-72a4-4d13-9074-3321fed6cc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:34.897195 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e35679-72a4-4d13-9074-3321fed6cc1e admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:34.897994 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e35679-72a4-4d13-9074-3321fed6cc1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:34.900117 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e35679-72a4-4d13-9074-3321fed6cc1e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:35.044411 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1355/2717] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:34 2022] POST /identity/v3/users => generated 300 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:35.048517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e39bfd-ef71-4957-adc3-10889e66388a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:35.058962 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e39bfd-ef71-4957-adc3-10889e66388a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:35.063648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36fc23-c073-43eb-8fa8-05831f01a65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:35.064435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36fc23-c073-43eb-8fa8-05831f01a65e admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:35.064585 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36fc23-c073-43eb-8fa8-05831f01a65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:35.064865 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36fc23-c073-43eb-8fa8-05831f01a65e admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:35.065566 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36fc23-c073-43eb-8fa8-05831f01a65e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:35.067160 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36fc23-c073-43eb-8fa8-05831f01a65e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:35.077103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1363/2718] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:35.081538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e35679-72a4-4d13-9074-3321fed6cc1e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:35.091495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e35679-72a4-4d13-9074-3321fed6cc1e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:35.096590 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001260d-78bd-4a31-b2d6-e815fbf5e1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:35.102048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001260d-78bd-4a31-b2d6-e815fbf5e1d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:35.102048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001260d-78bd-4a31-b2d6-e815fbf5e1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:35.102048 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001260d-78bd-4a31-b2d6-e815fbf5e1d7 admin admin] PATH_INFO: `/v3/projects/caa5290dafcf4d26a6c92e80ae7ffd5c/users/ff90267a8a824132bc66860d53a15002/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:35.114024 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001260d-78bd-4a31-b2d6-e815fbf5e1d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa5290dafcf4d26a6c92e80ae7ffd5c, user_id=ff90267a8a824132bc66860d53a15002, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:35.116752 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001260d-78bd-4a31-b2d6-e815fbf5e1d7 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:35.186730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1356/2719] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:35 2022] PUT /identity/v3/projects/caa5290dafcf4d26a6c92e80ae7ffd5c/users/ff90267a8a824132bc66860d53a15002/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:35.190710 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36fc23-c073-43eb-8fa8-05831f01a65e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:35.200764 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36fc23-c073-43eb-8fa8-05831f01a65e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:35.257245 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1e464-3638-43c6-980e-959b7ed9a531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:35.257993 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1e464-3638-43c6-980e-959b7ed9a531 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:35.258214 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1e464-3638-43c6-980e-959b7ed9a531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:35.258493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1e464-3638-43c6-980e-959b7ed9a531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:35.259209 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1e464-3638-43c6-980e-959b7ed9a531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:35.260965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1e464-3638-43c6-980e-959b7ed9a531 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:35.271886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1364/2720] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:35.275988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001260d-78bd-4a31-b2d6-e815fbf5e1d7 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:35.295370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001260d-78bd-4a31-b2d6-e815fbf5e1d7 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:35.304172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af99e8-b358-4746-89c3-57c65f60d7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:35.304978 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af99e8-b358-4746-89c3-57c65f60d7cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:35.305373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af99e8-b358-4746-89c3-57c65f60d7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:35.305853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af99e8-b358-4746-89c3-57c65f60d7cc admin admin] PATH_INFO: `/v3/projects/caa5290dafcf4d26a6c92e80ae7ffd5c/users/ff90267a8a824132bc66860d53a15002/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:35.307988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af99e8-b358-4746-89c3-57c65f60d7cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa5290dafcf4d26a6c92e80ae7ffd5c, user_id=ff90267a8a824132bc66860d53a15002, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:35.312797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af99e8-b358-4746-89c3-57c65f60d7cc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:35.325829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1357/2721] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:35 2022] PUT /identity/v3/projects/caa5290dafcf4d26a6c92e80ae7ffd5c/users/ff90267a8a824132bc66860d53a15002/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:35.337360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415c958-2e17-4d80-ad23-d3e77579180f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:35.340595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415c958-2e17-4d80-ad23-d3e77579180f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:35.340595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415c958-2e17-4d80-ad23-d3e77579180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:35.367314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8415c958-2e17-4d80-ad23-d3e77579180f None None] MFA Rules not processed for user `ff90267a8a824132bc66860d53a15002`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:35.370408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8415c958-2e17-4d80-ad23-d3e77579180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:35.438298 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1365/2722] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:35.449473 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af99e8-b358-4746-89c3-57c65f60d7cc admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:35.511494 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:35.515636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af99e8-b358-4746-89c3-57c65f60d7cc admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:35.519226 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80ac83-3e12-40c7-8e84-689ee9b342ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:35.519868 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80ac83-3e12-40c7-8e84-689ee9b342ac service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:35.520150 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80ac83-3e12-40c7-8e84-689ee9b342ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:35.520423 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80ac83-3e12-40c7-8e84-689ee9b342ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:35.529926 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80ac83-3e12-40c7-8e84-689ee9b342ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:35.531357 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80ac83-3e12-40c7-8e84-689ee9b342ac service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:35.540742 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1358/2723] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:35.610541 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415c958-2e17-4d80-ad23-d3e77579180f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:35.620264 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:35.671593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415c958-2e17-4d80-ad23-d3e77579180f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:35.676293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:35.676924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:35.677202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:35.677472 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] PATH_INFO: `/v3/users/14e6abe7564047f7ac377229f01704ac` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:35.678657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e6abe7564047f7ac377229f01704ac)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:35.680615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:35.874246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e6abe7564047f7ac377229f01704ac'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:35.880998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e6abe7564047f7ac377229f01704ac'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:35.968711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e6abe7564047f7ac377229f01704ac'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:35.969362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e6abe7564047f7ac377229f01704ac'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:35.981908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e6abe7564047f7ac377229f01704ac'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:35.989004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1366/2724] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:35 2022] DELETE /identity/v3/users/14e6abe7564047f7ac377229f01704ac => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:35.993598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd1108-4a13-44ad-94f7-d588368dc3b3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:35.994064 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd1108-4a13-44ad-94f7-d588368dc3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:35.994176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd1108-4a13-44ad-94f7-d588368dc3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:36.014623 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-7dfd1108-4a13-44ad-94f7-d588368dc3b3 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:36.015753 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7dfd1108-4a13-44ad-94f7-d588368dc3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:36.073353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1359/2725] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:36.087552 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:36.089031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:36.148587 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3451a-5f6e-4220-94ef-6d0d86936a8b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:36.152226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5454e-16f4-4790-8356-14c14b43d24a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:36.152842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5454e-16f4-4790-8356-14c14b43d24a service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:36.153125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5454e-16f4-4790-8356-14c14b43d24a service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:36.153400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5454e-16f4-4790-8356-14c14b43d24a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:36.162973 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5454e-16f4-4790-8356-14c14b43d24a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:36.164436 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5454e-16f4-4790-8356-14c14b43d24a service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:36.174454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1367/2726] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:36.582189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd1108-4a13-44ad-94f7-d588368dc3b3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:36.583659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7dfd1108-4a13-44ad-94f7-d588368dc3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:36.596487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd1108-4a13-44ad-94f7-d588368dc3b3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:36.601766 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:36.602492 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:36.602779 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:36.603054 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] PATH_INFO: `/v3/projects/c12c2059199042bcaaaf1d4131483812` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:36.604521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12c2059199042bcaaaf1d4131483812)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:36.606479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:36.711069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12c2059199042bcaaaf1d4131483812', 'request_id': 'req-c369806d-9b14-4ce9-9a61-0b757fa369c9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:36.716609 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12c2059199042bcaaaf1d4131483812', 'request_id': 'req-c369806d-9b14-4ce9-9a61-0b757fa369c9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:36.768327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12c2059199042bcaaaf1d4131483812', 'request_id': 'req-c369806d-9b14-4ce9-9a61-0b757fa369c9'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:36.779047 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] The token cache is being invalidate because project c12c2059199042bcaaaf1d4131483812 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:36.784854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:36.784854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:36.784854 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1360/2727] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:36 2022] DELETE /identity/v3/projects/c12c2059199042bcaaaf1d4131483812 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:36.785326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5454e-16f4-4790-8356-14c14b43d24a service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:36.794652 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5454e-16f4-4790-8356-14c14b43d24a service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:36.824954 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:36.852072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:36.852714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:36.853021 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:36.853295 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] PATH_INFO: `/v3/users/ff90267a8a824132bc66860d53a15002` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:36.854508 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff90267a8a824132bc66860d53a15002)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:36.856429 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:37.064842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff90267a8a824132bc66860d53a15002'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:37.071482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff90267a8a824132bc66860d53a15002'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:37.151260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff90267a8a824132bc66860d53a15002'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:37.151850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff90267a8a824132bc66860d53a15002'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:37.164314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-0b140591-8258-4305-994c-f7d09ed0cf21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff90267a8a824132bc66860d53a15002'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:37.170568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1368/2728] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:36 2022] DELETE /identity/v3/users/ff90267a8a824132bc66860d53a15002 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:37.552722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:37.554164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:37.567118 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369806d-9b14-4ce9-9a61-0b757fa369c9 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:37.633626 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:37.634268 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:37.634577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:37.634849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] PATH_INFO: `/v3/projects/caa5290dafcf4d26a6c92e80ae7ffd5c` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:37.636041 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa5290dafcf4d26a6c92e80ae7ffd5c)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:37.638013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:37.656440 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:37.734129 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa5290dafcf4d26a6c92e80ae7ffd5c', 'request_id': 'req-0e86ead3-4e14-45d1-a6ae-9a00611452d6'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:37.739621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa5290dafcf4d26a6c92e80ae7ffd5c', 'request_id': 'req-0e86ead3-4e14-45d1-a6ae-9a00611452d6'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:37.788442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa5290dafcf4d26a6c92e80ae7ffd5c', 'request_id': 'req-0e86ead3-4e14-45d1-a6ae-9a00611452d6'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:37.800270 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] The token cache is being invalidate because project caa5290dafcf4d26a6c92e80ae7ffd5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:37.800561 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:37.801189 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0e86ead3-4e14-45d1-a6ae-9a00611452d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:37.802694 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1361/2729] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:37 2022] DELETE /identity/v3/projects/caa5290dafcf4d26a6c92e80ae7ffd5c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:37.807516 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4662234-4f5f-4666-b1b0-fcbd97319e97 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:37.808007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4662234-4f5f-4666-b1b0-fcbd97319e97 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:37.808129 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4662234-4f5f-4666-b1b0-fcbd97319e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:37.827550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b4662234-4f5f-4666-b1b0-fcbd97319e97 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:37.828771 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b4662234-4f5f-4666-b1b0-fcbd97319e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:37.889214 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1369/2730] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:37.897426 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e881f-0da6-41b5-a3eb-2f82dbcf94ef None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:37.897723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e881f-0da6-41b5-a3eb-2f82dbcf94ef None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:37.898004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e881f-0da6-41b5-a3eb-2f82dbcf94ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:37.917327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e30e881f-0da6-41b5-a3eb-2f82dbcf94ef None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:37.918622 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e30e881f-0da6-41b5-a3eb-2f82dbcf94ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:37.931676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1362/2731] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:37.936383 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4662234-4f5f-4666-b1b0-fcbd97319e97 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:37.945959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4662234-4f5f-4666-b1b0-fcbd97319e97 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:37.950415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e152ab-1ba2-4493-8010-4d1ef663042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:37.951739 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e152ab-1ba2-4493-8010-4d1ef663042d admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:37.952066 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e152ab-1ba2-4493-8010-4d1ef663042d admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:37.952381 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e152ab-1ba2-4493-8010-4d1ef663042d admin admin] PATH_INFO: `/v3/domains` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:37.953326 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e152ab-1ba2-4493-8010-4d1ef663042d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80525) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:30:37.953827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e152ab-1ba2-4493-8010-4d1ef663042d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:37.956413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e152ab-1ba2-4493-8010-4d1ef663042d admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:37.971013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1370/2732] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:30:37.977980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e881f-0da6-41b5-a3eb-2f82dbcf94ef None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:37.987795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e881f-0da6-41b5-a3eb-2f82dbcf94ef None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:37.992441 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd941cd-c802-4aaf-90db-369c4eb4bfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:37.993046 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd941cd-c802-4aaf-90db-369c4eb4bfb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:37.993327 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd941cd-c802-4aaf-90db-369c4eb4bfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:37.993596 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd941cd-c802-4aaf-90db-369c4eb4bfb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:37.994267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd941cd-c802-4aaf-90db-369c4eb4bfb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:37.996192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd941cd-c802-4aaf-90db-369c4eb4bfb5 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:38.074658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1363/2733] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:37 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:38.079661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e152ab-1ba2-4493-8010-4d1ef663042d admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:38.089979 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e152ab-1ba2-4493-8010-4d1ef663042d admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:38.124484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:38.147767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef745f-bd30-4c52-92dd-9e620993ffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:38.148411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef745f-bd30-4c52-92dd-9e620993ffa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:38.148691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef745f-bd30-4c52-92dd-9e620993ffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:38.148965 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef745f-bd30-4c52-92dd-9e620993ffa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:38.149620 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef745f-bd30-4c52-92dd-9e620993ffa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:38.151550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef745f-bd30-4c52-92dd-9e620993ffa5 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:38.257191 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1371/2734] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:38 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:38.262634 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd941cd-c802-4aaf-90db-369c4eb4bfb5 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:38.282276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd941cd-c802-4aaf-90db-369c4eb4bfb5 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:38.289665 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca8a98-bb18-4dbe-bfcc-b5de9c64e051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:38.290303 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca8a98-bb18-4dbe-bfcc-b5de9c64e051 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:38.290587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca8a98-bb18-4dbe-bfcc-b5de9c64e051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:38.290867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca8a98-bb18-4dbe-bfcc-b5de9c64e051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:38.291582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca8a98-bb18-4dbe-bfcc-b5de9c64e051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:38.293319 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca8a98-bb18-4dbe-bfcc-b5de9c64e051 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:38.309276 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1364/2735] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:38.315608 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef745f-bd30-4c52-92dd-9e620993ffa5 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:38.327151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef745f-bd30-4c52-92dd-9e620993ffa5 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:38.331990 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b8814-0219-43ea-ac19-1ffc44093093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:38.332665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b8814-0219-43ea-ac19-1ffc44093093 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:38.332949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b8814-0219-43ea-ac19-1ffc44093093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:38.333226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b8814-0219-43ea-ac19-1ffc44093093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:38.333915 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b8814-0219-43ea-ac19-1ffc44093093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:38.335478 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b8814-0219-43ea-ac19-1ffc44093093 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:38.346063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1372/2736] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:38.349922 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca8a98-bb18-4dbe-bfcc-b5de9c64e051 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:38.360894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca8a98-bb18-4dbe-bfcc-b5de9c64e051 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:38.365469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22163c31-703e-4c41-b451-06ab29c70fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:38.366122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22163c31-703e-4c41-b451-06ab29c70fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:38.366402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22163c31-703e-4c41-b451-06ab29c70fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:38.366676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22163c31-703e-4c41-b451-06ab29c70fd3 admin admin] PATH_INFO: `/v3/projects/f596ab4ab27f4a7cb782f8bb8b2a4bb2/users/f63e379f3fad4650b949dc499a0661ff/roles/512a743b8ca748f8b793681748be7906` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:38.382550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22163c31-703e-4c41-b451-06ab29c70fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f596ab4ab27f4a7cb782f8bb8b2a4bb2, user_id=f63e379f3fad4650b949dc499a0661ff, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:38.385140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22163c31-703e-4c41-b451-06ab29c70fd3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:38.448586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1365/2737] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:38 2022] PUT /identity/v3/projects/f596ab4ab27f4a7cb782f8bb8b2a4bb2/users/f63e379f3fad4650b949dc499a0661ff/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:38.459188 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16ffe-05c5-44da-8ad8-a4831408b509 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:38.459475 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16ffe-05c5-44da-8ad8-a4831408b509 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:38.459772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16ffe-05c5-44da-8ad8-a4831408b509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:38.488161 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-40b16ffe-05c5-44da-8ad8-a4831408b509 None None] MFA Rules not processed for user `f63e379f3fad4650b949dc499a0661ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:38.489440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-40b16ffe-05c5-44da-8ad8-a4831408b509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:38.553537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1373/2738] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:38.557989 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22163c31-703e-4c41-b451-06ab29c70fd3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:38.619682 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22163c31-703e-4c41-b451-06ab29c70fd3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:38.625416 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b5d65-8345-41aa-a29d-870405572045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:38.626057 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b5d65-8345-41aa-a29d-870405572045 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:38.626368 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b5d65-8345-41aa-a29d-870405572045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:38.626775 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b5d65-8345-41aa-a29d-870405572045 admin admin] PATH_INFO: `/v3/projects` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:38.627456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b5d65-8345-41aa-a29d-870405572045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:38.629484 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b5d65-8345-41aa-a29d-870405572045 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:38.718648 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1366/2739] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:38 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:38.722722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b16ffe-05c5-44da-8ad8-a4831408b509 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:38.734148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b16ffe-05c5-44da-8ad8-a4831408b509 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:38.789602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a6423-0ed6-495e-a3eb-ce8417f728c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:38.790221 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a6423-0ed6-495e-a3eb-ce8417f728c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:38.790503 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a6423-0ed6-495e-a3eb-ce8417f728c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:38.790781 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a6423-0ed6-495e-a3eb-ce8417f728c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:38.791454 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a6423-0ed6-495e-a3eb-ce8417f728c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:38.793394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a6423-0ed6-495e-a3eb-ce8417f728c3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:38.880231 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1374/2740] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:38 2022] POST /identity/v3/users => generated 295 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:38.884200 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b5d65-8345-41aa-a29d-870405572045 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:38.894373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b5d65-8345-41aa-a29d-870405572045 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:38.899012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bea11-7676-4516-be35-eb3696471eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:38.899616 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bea11-7676-4516-be35-eb3696471eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:38.899956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bea11-7676-4516-be35-eb3696471eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:38.907066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bea11-7676-4516-be35-eb3696471eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:38.907066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26bea11-7676-4516-be35-eb3696471eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:38.907066 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26bea11-7676-4516-be35-eb3696471eb8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:38.914322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1367/2741] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:38.918658 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a6423-0ed6-495e-a3eb-ce8417f728c3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:38.929680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a6423-0ed6-495e-a3eb-ce8417f728c3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:38.934427 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d7fe0-bd4f-4da0-8516-17b3142a17d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:38.935089 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d7fe0-bd4f-4da0-8516-17b3142a17d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:38.935372 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d7fe0-bd4f-4da0-8516-17b3142a17d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:38.935646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d7fe0-bd4f-4da0-8516-17b3142a17d9 admin admin] PATH_INFO: `/v3/projects/0734518179f54ab2a88f096938db5af7/users/17d4bd9c395b4331a4229d0d67d568b4/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:38.952211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d7fe0-bd4f-4da0-8516-17b3142a17d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0734518179f54ab2a88f096938db5af7, user_id=17d4bd9c395b4331a4229d0d67d568b4, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:38.954903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d7fe0-bd4f-4da0-8516-17b3142a17d9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:39.023269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1375/2742] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:38 2022] PUT /identity/v3/projects/0734518179f54ab2a88f096938db5af7/users/17d4bd9c395b4331a4229d0d67d568b4/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:39.027281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bea11-7676-4516-be35-eb3696471eb8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:39.037763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bea11-7676-4516-be35-eb3696471eb8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:39.049120 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:39.098969 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36186864-adae-43e0-8ea7-e24612c87ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:39.099601 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36186864-adae-43e0-8ea7-e24612c87ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:39.099909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36186864-adae-43e0-8ea7-e24612c87ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:39.100211 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36186864-adae-43e0-8ea7-e24612c87ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:39.100880 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36186864-adae-43e0-8ea7-e24612c87ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:39.102513 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36186864-adae-43e0-8ea7-e24612c87ba8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:39.115842 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1368/2743] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:39.119476 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d7fe0-bd4f-4da0-8516-17b3142a17d9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:39.132623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d7fe0-bd4f-4da0-8516-17b3142a17d9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:39.138702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f51f0-f2f6-4827-b54d-4a9eba7f5700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:39.140076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f51f0-f2f6-4827-b54d-4a9eba7f5700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:39.140365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f51f0-f2f6-4827-b54d-4a9eba7f5700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:39.140649 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f51f0-f2f6-4827-b54d-4a9eba7f5700 admin admin] PATH_INFO: `/v3/projects/0734518179f54ab2a88f096938db5af7/users/17d4bd9c395b4331a4229d0d67d568b4/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:39.142647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f51f0-f2f6-4827-b54d-4a9eba7f5700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0734518179f54ab2a88f096938db5af7, user_id=17d4bd9c395b4331a4229d0d67d568b4, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:39.145434 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f51f0-f2f6-4827-b54d-4a9eba7f5700 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:39.151345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:39.157969 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1376/2744] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:39 2022] PUT /identity/v3/projects/0734518179f54ab2a88f096938db5af7/users/17d4bd9c395b4331a4229d0d67d568b4/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:39.225882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7fd14-d135-4568-a9db-922fe7ab8430 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:39.226169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7fd14-d135-4568-a9db-922fe7ab8430 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:39.226442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7fd14-d135-4568-a9db-922fe7ab8430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:39.260088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-efb7fd14-d135-4568-a9db-922fe7ab8430 None None] MFA Rules not processed for user `17d4bd9c395b4331a4229d0d67d568b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:39.261372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-efb7fd14-d135-4568-a9db-922fe7ab8430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:39.334210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1369/2745] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:39.347296 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f51f0-f2f6-4827-b54d-4a9eba7f5700 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:39.349292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-584f51f0-f2f6-4827-b54d-4a9eba7f5700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:39.409985 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f51f0-f2f6-4827-b54d-4a9eba7f5700 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:39.413863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af3a00-c35d-46d8-ac61-9b8312620a9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:39.414550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af3a00-c35d-46d8-ac61-9b8312620a9f service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:39.414832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af3a00-c35d-46d8-ac61-9b8312620a9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:39.415105 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af3a00-c35d-46d8-ac61-9b8312620a9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:39.432059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af3a00-c35d-46d8-ac61-9b8312620a9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:39.433650 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af3a00-c35d-46d8-ac61-9b8312620a9f service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:39.457246 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1377/2746] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:40.590381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7fd14-d135-4568-a9db-922fe7ab8430 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:40.601086 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7fd14-d135-4568-a9db-922fe7ab8430 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:40.606254 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac10254-b4c0-4b7c-bea7-9faef79adaf7 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d4bd9c395b4331a4229d0d67d568b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0734518179f54ab2a88f096938db5af7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:40.607000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac10254-b4c0-4b7c-bea7-9faef79adaf7 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:40.607289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac10254-b4c0-4b7c-bea7-9faef79adaf7 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:40.607566 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac10254-b4c0-4b7c-bea7-9faef79adaf7 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:40.608373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac10254-b4c0-4b7c-bea7-9faef79adaf7 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:30:40.608686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac10254-b4c0-4b7c-bea7-9faef79adaf7 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:40.611016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac10254-b4c0-4b7c-bea7-9faef79adaf7 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:40.625916 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1370/2747] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:30:40.635134 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af3a00-c35d-46d8-ac61-9b8312620a9f service cinder] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:40.642665 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af3a00-c35d-46d8-ac61-9b8312620a9f service cinder] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:40.647440 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bf094-2772-425c-b556-ebcb24b6247c tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d4bd9c395b4331a4229d0d67d568b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0734518179f54ab2a88f096938db5af7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:40.648114 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bf094-2772-425c-b556-ebcb24b6247c tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:40.648401 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bf094-2772-425c-b556-ebcb24b6247c tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:40.648684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bf094-2772-425c-b556-ebcb24b6247c tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:40.649371 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bf094-2772-425c-b556-ebcb24b6247c tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:40.652697 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bf094-2772-425c-b556-ebcb24b6247c tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:40.770025 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1378/2748] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:40 2022] POST /identity/v3/projects => generated 354 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:40.773664 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac10254-b4c0-4b7c-bea7-9faef79adaf7 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:40.787712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac10254-b4c0-4b7c-bea7-9faef79adaf7 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:40.850636 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023983d3-a67b-4582-ba66-b36acc8d5ff0 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d4bd9c395b4331a4229d0d67d568b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0734518179f54ab2a88f096938db5af7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:40.851376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023983d3-a67b-4582-ba66-b36acc8d5ff0 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:40.851823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023983d3-a67b-4582-ba66-b36acc8d5ff0 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:40.852095 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023983d3-a67b-4582-ba66-b36acc8d5ff0 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:40.852798 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023983d3-a67b-4582-ba66-b36acc8d5ff0 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:30:40.855453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023983d3-a67b-4582-ba66-b36acc8d5ff0 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:40.855453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023983d3-a67b-4582-ba66-b36acc8d5ff0 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:40.868910 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1371/2749] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:30:40.911026 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bf094-2772-425c-b556-ebcb24b6247c tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:40.923570 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bf094-2772-425c-b556-ebcb24b6247c tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:40.928789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d4bd9c395b4331a4229d0d67d568b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0734518179f54ab2a88f096938db5af7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:40.929591 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:40.929880 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:40.930250 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] PATH_INFO: `/v3/projects/4f65b65016fb46219e51f6cd0848982b` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:40.931572 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4f65b65016fb46219e51f6cd0848982b)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:40.933418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:41.054600 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f65b65016fb46219e51f6cd0848982b', 'request_id': 'req-7d7464d4-8836-48a5-8a1d-bcd1838e05da'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:41.062374 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f65b65016fb46219e51f6cd0848982b', 'request_id': 'req-7d7464d4-8836-48a5-8a1d-bcd1838e05da'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:41.068453 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f65b65016fb46219e51f6cd0848982b', 'request_id': 'req-7d7464d4-8836-48a5-8a1d-bcd1838e05da'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:41.156382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] The token cache is being invalidate because project 4f65b65016fb46219e51f6cd0848982b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:41.156382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:41.157193 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-7d7464d4-8836-48a5-8a1d-bcd1838e05da tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:41.159411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1379/2750] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:40 2022] DELETE /identity/v3/projects/4f65b65016fb46219e51f6cd0848982b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:30:41.828655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023983d3-a67b-4582-ba66-b36acc8d5ff0 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:41.830588 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-023983d3-a67b-4582-ba66-b36acc8d5ff0 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:41.851345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:41.899010 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023983d3-a67b-4582-ba66-b36acc8d5ff0 tempest-VolumeQuotaClassesTest-54850230 tempest-VolumeQuotaClassesTest-54850230-project-admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:41.955405 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:41.956385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:41.956675 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:41.956954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] PATH_INFO: `/v3/users/f63e379f3fad4650b949dc499a0661ff` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:41.958192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f63e379f3fad4650b949dc499a0661ff)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:41.960187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:42.141705 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f63e379f3fad4650b949dc499a0661ff'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:42.148829 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f63e379f3fad4650b949dc499a0661ff'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:42.225669 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f63e379f3fad4650b949dc499a0661ff'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:42.226376 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f63e379f3fad4650b949dc499a0661ff'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:42.227115 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f63e379f3fad4650b949dc499a0661ff'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:42.247235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1372/2751] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:41 2022] DELETE /identity/v3/users/f63e379f3fad4650b949dc499a0661ff => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:42.252509 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ea273-e6fa-4981-88ea-70d43d155669 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:42.252999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ea273-e6fa-4981-88ea-70d43d155669 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:42.253102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ea273-e6fa-4981-88ea-70d43d155669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:42.273798 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-526ea273-e6fa-4981-88ea-70d43d155669 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:42.275013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-526ea273-e6fa-4981-88ea-70d43d155669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:42.347590 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1380/2752] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:42.365122 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:42.366735 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:42.425529 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:42.436712 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cfe9d-f3bd-480e-acff-c8e8d3ac24dc admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:42.440941 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85945b2-3d76-4e55-a3c2-31916285d510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:42.441138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85945b2-3d76-4e55-a3c2-31916285d510 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:42.441500 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85945b2-3d76-4e55-a3c2-31916285d510 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:42.441785 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85945b2-3d76-4e55-a3c2-31916285d510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:42.452852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85945b2-3d76-4e55-a3c2-31916285d510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:42.454369 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85945b2-3d76-4e55-a3c2-31916285d510 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:42.464937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1373/2753] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:42.896151 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ea273-e6fa-4981-88ea-70d43d155669 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:42.897707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-526ea273-e6fa-4981-88ea-70d43d155669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:42.908834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:42.913827 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ea273-e6fa-4981-88ea-70d43d155669 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:42.918883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:42.919534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:42.919851 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:42.920170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] PATH_INFO: `/v3/projects/f596ab4ab27f4a7cb782f8bb8b2a4bb2` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:42.921388 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f596ab4ab27f4a7cb782f8bb8b2a4bb2)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:42.924989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:43.089544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f596ab4ab27f4a7cb782f8bb8b2a4bb2', 'request_id': 'req-505328bf-f687-4bd0-bc14-07e2edca85cd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:43.096542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f596ab4ab27f4a7cb782f8bb8b2a4bb2', 'request_id': 'req-505328bf-f687-4bd0-bc14-07e2edca85cd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:43.104302 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f596ab4ab27f4a7cb782f8bb8b2a4bb2', 'request_id': 'req-505328bf-f687-4bd0-bc14-07e2edca85cd'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:43.214252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] The token cache is being invalidate because project f596ab4ab27f4a7cb782f8bb8b2a4bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:43.214841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:43.215648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-505328bf-f687-4bd0-bc14-07e2edca85cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:43.217533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1381/2754] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:42 2022] DELETE /identity/v3/projects/f596ab4ab27f4a7cb782f8bb8b2a4bb2 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:43.221632 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85945b2-3d76-4e55-a3c2-31916285d510 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:43.231901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85945b2-3d76-4e55-a3c2-31916285d510 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:43.289229 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:43.289875 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:43.290161 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:43.290435 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] PATH_INFO: `/v3/users/17d4bd9c395b4331a4229d0d67d568b4` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:43.291693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d4bd9c395b4331a4229d0d67d568b4)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:43.293674 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:43.485026 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ba6a6-7d9f-42e0-ad58-0c08abaaf3d6 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:43.485545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ba6a6-7d9f-42e0-ad58-0c08abaaf3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:43.485545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ba6a6-7d9f-42e0-ad58-0c08abaaf3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:43.507482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-fb2ba6a6-7d9f-42e0-ad58-0c08abaaf3d6 None None] MFA Rules not processed for user `42047fe93eb24adbbc24844a75ed21f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:43.508723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fb2ba6a6-7d9f-42e0-ad58-0c08abaaf3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:43.566042 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1382/2755] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:43.613037 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d4bd9c395b4331a4229d0d67d568b4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:43.622587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d4bd9c395b4331a4229d0d67d568b4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:43.711173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d4bd9c395b4331a4229d0d67d568b4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:43.711886 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d4bd9c395b4331a4229d0d67d568b4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:43.712509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-0100b6eb-1386-4662-b850-c478e85b76d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d4bd9c395b4331a4229d0d67d568b4'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:43.735828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1374/2756] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:30:43 2022] DELETE /identity/v3/users/17d4bd9c395b4331a4229d0d67d568b4 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:44.188113 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ba6a6-7d9f-42e0-ad58-0c08abaaf3d6 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:44.189906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fb2ba6a6-7d9f-42e0-ad58-0c08abaaf3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:44.216568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ba6a6-7d9f-42e0-ad58-0c08abaaf3d6 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:44.293054 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:44.293711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:44.294004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:44.294418 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] PATH_INFO: `/v3/projects/0734518179f54ab2a88f096938db5af7` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:44.295691 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0734518179f54ab2a88f096938db5af7)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:44.297692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:44.431616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:44.449102 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0734518179f54ab2a88f096938db5af7', 'request_id': 'req-e9916bb6-1861-4ea0-ba67-09e692631a02'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:44.455902 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0734518179f54ab2a88f096938db5af7', 'request_id': 'req-e9916bb6-1861-4ea0-ba67-09e692631a02'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:44.461272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0734518179f54ab2a88f096938db5af7', 'request_id': 'req-e9916bb6-1861-4ea0-ba67-09e692631a02'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:44.593891 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] The token cache is being invalidate because project 0734518179f54ab2a88f096938db5af7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:30:44.594617 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:44.596205 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-e9916bb6-1861-4ea0-ba67-09e692631a02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:30:44.599140 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1383/2757] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:30:44 2022] DELETE /identity/v3/projects/0734518179f54ab2a88f096938db5af7 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:44.604533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28283a2-190c-4c6e-bab0-49ef43e8684c None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:44.605322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28283a2-190c-4c6e-bab0-49ef43e8684c None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:44.605604 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28283a2-190c-4c6e-bab0-49ef43e8684c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:44.626402 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b28283a2-190c-4c6e-bab0-49ef43e8684c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:44.627607 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b28283a2-190c-4c6e-bab0-49ef43e8684c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:44.708156 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1375/2758] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:44.709629 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60446863-8bee-4d3a-82b4-40e195728293 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:44.709923 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60446863-8bee-4d3a-82b4-40e195728293 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:44.710202 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60446863-8bee-4d3a-82b4-40e195728293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:44.734956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-60446863-8bee-4d3a-82b4-40e195728293 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:44.736901 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-60446863-8bee-4d3a-82b4-40e195728293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:44.754657 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1384/2759] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:44.756164 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28283a2-190c-4c6e-bab0-49ef43e8684c None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:44.768325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28283a2-190c-4c6e-bab0-49ef43e8684c None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:44.776318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13aed66-1665-4ac6-9a37-3b6f3e93b362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:44.777186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13aed66-1665-4ac6-9a37-3b6f3e93b362 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:44.777638 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13aed66-1665-4ac6-9a37-3b6f3e93b362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:44.778109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13aed66-1665-4ac6-9a37-3b6f3e93b362 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:44.779011 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13aed66-1665-4ac6-9a37-3b6f3e93b362 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:30:44.779520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13aed66-1665-4ac6-9a37-3b6f3e93b362 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:44.781398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13aed66-1665-4ac6-9a37-3b6f3e93b362 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:44.783192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:44.796158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1376/2760] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:30:44.800071 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60446863-8bee-4d3a-82b4-40e195728293 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:44.819180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60446863-8bee-4d3a-82b4-40e195728293 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:44.825405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df604e2-4e93-46ff-9182-915f30248ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:44.826425 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df604e2-4e93-46ff-9182-915f30248ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:44.826761 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df604e2-4e93-46ff-9182-915f30248ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:44.828000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df604e2-4e93-46ff-9182-915f30248ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:44.828770 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df604e2-4e93-46ff-9182-915f30248ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:44.831297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df604e2-4e93-46ff-9182-915f30248ccf admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:44.938169 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1385/2761] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:30:44 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:30:44.942499 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13aed66-1665-4ac6-9a37-3b6f3e93b362 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:44.954749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13aed66-1665-4ac6-9a37-3b6f3e93b362 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:45.019373 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c85e7-fcc3-4944-a4b1-1abbb9fe912a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:45.020745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c85e7-fcc3-4944-a4b1-1abbb9fe912a admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:45.020745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c85e7-fcc3-4944-a4b1-1abbb9fe912a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:45.020745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c85e7-fcc3-4944-a4b1-1abbb9fe912a admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:45.021325 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c85e7-fcc3-4944-a4b1-1abbb9fe912a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:45.024530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c85e7-fcc3-4944-a4b1-1abbb9fe912a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:45.138079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1377/2762] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:30:44 2022] POST /identity/v3/users => generated 286 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:30:45.139312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df604e2-4e93-46ff-9182-915f30248ccf admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:45.151629 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df604e2-4e93-46ff-9182-915f30248ccf admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:45.157684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad28395-2cf4-4c46-809c-522b5c0409bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:45.158667 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad28395-2cf4-4c46-809c-522b5c0409bb admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:45.159009 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad28395-2cf4-4c46-809c-522b5c0409bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:45.160260 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad28395-2cf4-4c46-809c-522b5c0409bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:45.161033 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad28395-2cf4-4c46-809c-522b5c0409bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:45.162653 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad28395-2cf4-4c46-809c-522b5c0409bb admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:45.174076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1386/2763] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 01:30:45.180320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c85e7-fcc3-4944-a4b1-1abbb9fe912a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:45.192820 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c85e7-fcc3-4944-a4b1-1abbb9fe912a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:45.199347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac94ee4-5e1a-44dd-9e26-2790b5e86a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:45.200098 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac94ee4-5e1a-44dd-9e26-2790b5e86a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:45.200533 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac94ee4-5e1a-44dd-9e26-2790b5e86a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:45.200804 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac94ee4-5e1a-44dd-9e26-2790b5e86a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:45.201578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac94ee4-5e1a-44dd-9e26-2790b5e86a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:45.203282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac94ee4-5e1a-44dd-9e26-2790b5e86a69 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:45.216396 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1378/2764] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:30:45.220602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad28395-2cf4-4c46-809c-522b5c0409bb admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:45.230833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad28395-2cf4-4c46-809c-522b5c0409bb admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:45.236686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f775a-98fa-4e96-891e-c049341e2689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:45.237350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f775a-98fa-4e96-891e-c049341e2689 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:45.237632 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f775a-98fa-4e96-891e-c049341e2689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:45.237907 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f775a-98fa-4e96-891e-c049341e2689 admin admin] PATH_INFO: `/v3/projects/6f206a8e662b423aacbfa9e41f3ae417/users/edaee05fe2854ef3a490ebc4911f5049/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:45.254651 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f775a-98fa-4e96-891e-c049341e2689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f206a8e662b423aacbfa9e41f3ae417, user_id=edaee05fe2854ef3a490ebc4911f5049, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:45.257395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f775a-98fa-4e96-891e-c049341e2689 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:45.357181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1387/2765] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:30:45 2022] PUT /identity/v3/projects/6f206a8e662b423aacbfa9e41f3ae417/users/edaee05fe2854ef3a490ebc4911f5049/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:30:45.363541 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202020dc-01bc-42a5-8659-d3ffedfe096e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:45.363837 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202020dc-01bc-42a5-8659-d3ffedfe096e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:45.364243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202020dc-01bc-42a5-8659-d3ffedfe096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:45.385174 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-202020dc-01bc-42a5-8659-d3ffedfe096e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:45.386394 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-202020dc-01bc-42a5-8659-d3ffedfe096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:45.462032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1379/2766] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:45.476201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f775a-98fa-4e96-891e-c049341e2689 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:45.481316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-908f775a-98fa-4e96-891e-c049341e2689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:45.566595 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f775a-98fa-4e96-891e-c049341e2689 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:45.571031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9d59c-4419-479d-9fab-c7a304471647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:45.571678 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9d59c-4419-479d-9fab-c7a304471647 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:45.572059 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9d59c-4419-479d-9fab-c7a304471647 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:45.572405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9d59c-4419-479d-9fab-c7a304471647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:45.583218 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9d59c-4419-479d-9fab-c7a304471647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:45.584919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9d59c-4419-479d-9fab-c7a304471647 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:45.599884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1388/2767] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:47.027255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e9f3f-44cc-439a-97ad-413cd08e1496 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:47.032058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e9f3f-44cc-439a-97ad-413cd08e1496 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:47.032058 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e9f3f-44cc-439a-97ad-413cd08e1496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:47.085891 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ae3e9f3f-44cc-439a-97ad-413cd08e1496 None None] MFA Rules not processed for user `edaee05fe2854ef3a490ebc4911f5049`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:47.087063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ae3e9f3f-44cc-439a-97ad-413cd08e1496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:47.178418 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1380/2768] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:47.193772 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9d59c-4419-479d-9fab-c7a304471647 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:30:47.195568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-0df9d59c-4419-479d-9fab-c7a304471647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:47.289096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9d59c-4419-479d-9fab-c7a304471647 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:30:47.295415 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa10f58-3080-42d9-9900-a55bf371c80b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:30:47.296267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa10f58-3080-42d9-9900-a55bf371c80b service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:47.296644 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa10f58-3080-42d9-9900-a55bf371c80b service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:47.297018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa10f58-3080-42d9-9900-a55bf371c80b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:47.313199 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa10f58-3080-42d9-9900-a55bf371c80b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:30:47.315077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa10f58-3080-42d9-9900-a55bf371c80b service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:30:47.328016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1389/2769] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:30:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:30:51.172727 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b314d-155b-4bfb-bae6-66559366a069 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:30:51.173245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b314d-155b-4bfb-bae6-66559366a069 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:30:51.173245 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b314d-155b-4bfb-bae6-66559366a069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:30:51.182149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:30:51.197688 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-d16b314d-155b-4bfb-bae6-66559366a069 None None] MFA Rules not processed for user `edaee05fe2854ef3a490ebc4911f5049`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:30:51.199987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d16b314d-155b-4bfb-bae6-66559366a069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:30:51.207803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1381/2770] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:30:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:30:57.323115 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:31:01.399387 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=2 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:31:07.056424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a6ca6-3488-41e3-9441-00c75290061e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:07.056958 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a6ca6-3488-41e3-9441-00c75290061e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:07.057064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a6ca6-3488-41e3-9441-00c75290061e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:07.077030 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-722a6ca6-3488-41e3-9441-00c75290061e None None] MFA Rules not processed for user `f05648c4a067490693bd554f458c6a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:07.078603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-722a6ca6-3488-41e3-9441-00c75290061e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:07.133191 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1390/2771] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:31:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:31:10.527899 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b314d-155b-4bfb-bae6-66559366a069 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:31:10.547745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b314d-155b-4bfb-bae6-66559366a069 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:31:10.551494 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a32620-77ba-4436-a974-1983c57f278d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:31:10.552167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a32620-77ba-4436-a974-1983c57f278d service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:10.552453 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a32620-77ba-4436-a974-1983c57f278d service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:10.552728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a32620-77ba-4436-a974-1983c57f278d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:10.554511 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-05a32620-77ba-4436-a974-1983c57f278d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:10.622177 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a32620-77ba-4436-a974-1983c57f278d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:31:10.625306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a32620-77ba-4436-a974-1983c57f278d service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:31:10.635581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1382/2772] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:31:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:31:17.126358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=25 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:31:20.630971 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=34 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:31:23.441206 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0891642-e69b-4296-8178-fb8eda54bb7c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:23.441724 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0891642-e69b-4296-8178-fb8eda54bb7c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:23.441842 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0891642-e69b-4296-8178-fb8eda54bb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:23.460258 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-b0891642-e69b-4296-8178-fb8eda54bb7c None None] MFA Rules not processed for user `edaee05fe2854ef3a490ebc4911f5049`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:23.461550 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b0891642-e69b-4296-8178-fb8eda54bb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:23.469938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1391/2773] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:31:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:31:23.879827 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb2c3-a9ce-41d2-b73a-d341e4fa8d1f None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:23.883347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb2c3-a9ce-41d2-b73a-d341e4fa8d1f None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:23.883347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41abb2c3-a9ce-41d2-b73a-d341e4fa8d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:23.899406 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-41abb2c3-a9ce-41d2-b73a-d341e4fa8d1f None None] MFA Rules not processed for user `165812ae8a304c1a94bc62dd84144d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:23.900642 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-41abb2c3-a9ce-41d2-b73a-d341e4fa8d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:23.956628 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1383/2774] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:31:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:31:33.455143 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:31:33.949545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=25 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:31:42.343740 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0891642-e69b-4296-8178-fb8eda54bb7c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:31:42.345183 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b0891642-e69b-4296-8178-fb8eda54bb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:42.434384 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0891642-e69b-4296-8178-fb8eda54bb7c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:31:42.440039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea02450-ade7-4081-9c5f-c71437318759 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:31:42.440686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea02450-ade7-4081-9c5f-c71437318759 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:42.440967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea02450-ade7-4081-9c5f-c71437318759 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:42.441243 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea02450-ade7-4081-9c5f-c71437318759 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:42.443013 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7ea02450-ade7-4081-9c5f-c71437318759 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:42.455753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea02450-ade7-4081-9c5f-c71437318759 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:31:42.457530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea02450-ade7-4081-9c5f-c71437318759 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:31:42.468416 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1392/2775] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:31:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:31:50.845134 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418bca9-e985-4202-910f-de7abb01335d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:50.845703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418bca9-e985-4202-910f-de7abb01335d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:50.845703 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418bca9-e985-4202-910f-de7abb01335d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:50.879900 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-b418bca9-e985-4202-910f-de7abb01335d None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:31:50.882963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b418bca9-e985-4202-910f-de7abb01335d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:50.941946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1384/2776] 10.222.0.25 () {58 vars in 1009 bytes} [Sat Aug 6 01:31:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:31:51.080097 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea02450-ade7-4081-9c5f-c71437318759 service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:31:51.082081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-7ea02450-ade7-4081-9c5f-c71437318759 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:31:51.154998 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea02450-ade7-4081-9c5f-c71437318759 service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:31:51.160628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76363c6-419b-4eb1-b94d-9c155f533e5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:31:51.161251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76363c6-419b-4eb1-b94d-9c155f533e5b service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:31:51.161534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76363c6-419b-4eb1-b94d-9c155f533e5b service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:31:51.161814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76363c6-419b-4eb1-b94d-9c155f533e5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:31:51.169948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76363c6-419b-4eb1-b94d-9c155f533e5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:31:51.171525 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76363c6-419b-4eb1-b94d-9c155f533e5b service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:31:51.179593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1393/2777] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:31:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:32:00.935725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=32 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:01.176534 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=73 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:07.957216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e1cd-bc8e-4521-9e9a-349343236340 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:07.957696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e1cd-bc8e-4521-9e9a-349343236340 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:07.957814 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e1cd-bc8e-4521-9e9a-349343236340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:07.977168 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-8e72e1cd-bc8e-4521-9e9a-349343236340 None None] MFA Rules not processed for user `6d62ec1463c14665b94e9b2d85e48e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:07.978427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-8e72e1cd-bc8e-4521-9e9a-349343236340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:07.986088 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1385/2778] 10.222.0.25 () {58 vars in 967 bytes} [Sat Aug 6 01:32:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:07.998475 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76363c6-419b-4eb1-b94d-9c155f533e5b service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:08.009513 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76363c6-419b-4eb1-b94d-9c155f533e5b service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:08.014744 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba916103-50b7-4e46-a4fa-0b9967decc14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33aa626411f1474595102a4541af901a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:08.015349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba916103-50b7-4e46-a4fa-0b9967decc14 service swift] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:08.015636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba916103-50b7-4e46-a4fa-0b9967decc14 service swift] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:08.015959 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba916103-50b7-4e46-a4fa-0b9967decc14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:08.024022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba916103-50b7-4e46-a4fa-0b9967decc14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:08.025588 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba916103-50b7-4e46-a4fa-0b9967decc14 service swift] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:08.034018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1394/2779] 10.222.0.25 () {60 vars in 1384 bytes} [Sat Aug 6 01:32:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 01:32:10.944728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72e1cd-bc8e-4521-9e9a-349343236340 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:10.956655 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72e1cd-bc8e-4521-9e9a-349343236340 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:10.961348 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:10.961995 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:10.964849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:10.964849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] PATH_INFO: `/v3/users/edaee05fe2854ef3a490ebc4911f5049` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:10.964849 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=edaee05fe2854ef3a490ebc4911f5049)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:10.965647 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:11.174185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edaee05fe2854ef3a490ebc4911f5049'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:11.180927 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edaee05fe2854ef3a490ebc4911f5049'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:11.244279 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edaee05fe2854ef3a490ebc4911f5049'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:11.244917 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edaee05fe2854ef3a490ebc4911f5049'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:11.245532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-33ae1012-1cee-4374-8fb4-98e142255b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edaee05fe2854ef3a490ebc4911f5049'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:11.262776 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1386/2780] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:32:10 2022] DELETE /identity/v3/users/edaee05fe2854ef3a490ebc4911f5049 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:11.491822 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba916103-50b7-4e46-a4fa-0b9967decc14 service swift] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:11.502760 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba916103-50b7-4e46-a4fa-0b9967decc14 service swift] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:11.559751 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:11.560417 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:11.560702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:11.560978 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] PATH_INFO: `/v3/projects/6f206a8e662b423aacbfa9e41f3ae417` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:11.562125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f206a8e662b423aacbfa9e41f3ae417)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:11.564049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:11.661575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f206a8e662b423aacbfa9e41f3ae417', 'request_id': 'req-6cc34981-29e3-4262-a8ad-74157c805368'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:11.667722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f206a8e662b423aacbfa9e41f3ae417', 'request_id': 'req-6cc34981-29e3-4262-a8ad-74157c805368'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:11.673226 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f206a8e662b423aacbfa9e41f3ae417', 'request_id': 'req-6cc34981-29e3-4262-a8ad-74157c805368'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:11.761537 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] The token cache is being invalidate because project 6f206a8e662b423aacbfa9e41f3ae417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:32:11.761834 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:11.762456 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-6cc34981-29e3-4262-a8ad-74157c805368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:11.764117 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1395/2781] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:32:11 2022] DELETE /identity/v3/projects/6f206a8e662b423aacbfa9e41f3ae417 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:11.774149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1dc8f-7732-40a0-a7fb-d4bef5e3a386 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:11.774442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1dc8f-7732-40a0-a7fb-d4bef5e3a386 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:11.774722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1dc8f-7732-40a0-a7fb-d4bef5e3a386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:11.793946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e3e1dc8f-7732-40a0-a7fb-d4bef5e3a386 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:11.795148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e3e1dc8f-7732-40a0-a7fb-d4bef5e3a386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:11.852740 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1387/2782] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:11.860410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ceaf7-5a27-4931-bf2d-91da8e30032f None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:11.863683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ceaf7-5a27-4931-bf2d-91da8e30032f None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:11.863683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ceaf7-5a27-4931-bf2d-91da8e30032f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:11.881840 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-8e8ceaf7-5a27-4931-bf2d-91da8e30032f None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:11.883035 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-8e8ceaf7-5a27-4931-bf2d-91da8e30032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:11.890987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1396/2783] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:11.895222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1dc8f-7732-40a0-a7fb-d4bef5e3a386 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:11.904714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1dc8f-7732-40a0-a7fb-d4bef5e3a386 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:11.909333 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1da3c1-b74e-4d71-ac5a-f6c064a6aefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:11.909938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1da3c1-b74e-4d71-ac5a-f6c064a6aefe admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:11.910218 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1da3c1-b74e-4d71-ac5a-f6c064a6aefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:11.910545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1da3c1-b74e-4d71-ac5a-f6c064a6aefe admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:11.911179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1da3c1-b74e-4d71-ac5a-f6c064a6aefe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:32:11.911478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1da3c1-b74e-4d71-ac5a-f6c064a6aefe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:11.913166 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1da3c1-b74e-4d71-ac5a-f6c064a6aefe admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:11.924961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1388/2784] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:32:11.929142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ceaf7-5a27-4931-bf2d-91da8e30032f None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:11.938410 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ceaf7-5a27-4931-bf2d-91da8e30032f None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:11.943871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532178a-4604-4f2d-a71a-a46b2adda907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:11.944531 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532178a-4604-4f2d-a71a-a46b2adda907 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:11.944821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532178a-4604-4f2d-a71a-a46b2adda907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:11.945107 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532178a-4604-4f2d-a71a-a46b2adda907 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:11.945924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532178a-4604-4f2d-a71a-a46b2adda907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:11.947656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532178a-4604-4f2d-a71a-a46b2adda907 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:12.035850 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1397/2785] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:32:11 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:12.039430 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1da3c1-b74e-4d71-ac5a-f6c064a6aefe admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:12.049857 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1da3c1-b74e-4d71-ac5a-f6c064a6aefe admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:12.097876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:12.106815 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe8689-719b-4e23-a190-7b0815e29c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:12.107438 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe8689-719b-4e23-a190-7b0815e29c12 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:12.107720 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe8689-719b-4e23-a190-7b0815e29c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:12.108026 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe8689-719b-4e23-a190-7b0815e29c12 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:12.108695 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe8689-719b-4e23-a190-7b0815e29c12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:12.110977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe8689-719b-4e23-a190-7b0815e29c12 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:12.200840 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1389/2786] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:32:12 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:32:12.204832 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532178a-4604-4f2d-a71a-a46b2adda907 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:12.215180 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532178a-4604-4f2d-a71a-a46b2adda907 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:12.219800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606a21d-351b-4cfb-9f75-39def833a768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:12.220423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606a21d-351b-4cfb-9f75-39def833a768 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:12.220709 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606a21d-351b-4cfb-9f75-39def833a768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:12.221493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606a21d-351b-4cfb-9f75-39def833a768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:12.222240 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606a21d-351b-4cfb-9f75-39def833a768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:12.223845 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606a21d-351b-4cfb-9f75-39def833a768 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:12.233886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1398/2787] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:12.238666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe8689-719b-4e23-a190-7b0815e29c12 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:12.248459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe8689-719b-4e23-a190-7b0815e29c12 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:12.253176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bee55-70f3-4b98-bc84-221537ff4e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:12.253784 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bee55-70f3-4b98-bc84-221537ff4e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:12.254065 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bee55-70f3-4b98-bc84-221537ff4e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:12.254337 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bee55-70f3-4b98-bc84-221537ff4e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:12.255018 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bee55-70f3-4b98-bc84-221537ff4e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:12.256671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bee55-70f3-4b98-bc84-221537ff4e32 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:12.266929 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1390/2788] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:32:12.270910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606a21d-351b-4cfb-9f75-39def833a768 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:12.280892 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606a21d-351b-4cfb-9f75-39def833a768 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:12.286499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712f4c3-59f7-441f-9e8a-55c1aeb2b104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:12.286766 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712f4c3-59f7-441f-9e8a-55c1aeb2b104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:12.287050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712f4c3-59f7-441f-9e8a-55c1aeb2b104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:12.287329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712f4c3-59f7-441f-9e8a-55c1aeb2b104 admin admin] PATH_INFO: `/v3/projects/4f9f4a213d9c4e35b0fdd385c0aaf339/users/0d3efa570f0142d1963bacbc00971a62/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:12.317449 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712f4c3-59f7-441f-9e8a-55c1aeb2b104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9f4a213d9c4e35b0fdd385c0aaf339, user_id=0d3efa570f0142d1963bacbc00971a62, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:12.320211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712f4c3-59f7-441f-9e8a-55c1aeb2b104 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:12.420835 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1399/2789] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:32:12 2022] PUT /identity/v3/projects/4f9f4a213d9c4e35b0fdd385c0aaf339/users/0d3efa570f0142d1963bacbc00971a62/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:12.431904 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f99c-4ef1-4fb1-81fa-fc4bcb2c2ec3 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:12.432230 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f99c-4ef1-4fb1-81fa-fc4bcb2c2ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:12.432467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f99c-4ef1-4fb1-81fa-fc4bcb2c2ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:12.460192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5e63f99c-4ef1-4fb1-81fa-fc4bcb2c2ec3 None None] MFA Rules not processed for user `0d3efa570f0142d1963bacbc00971a62`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:12.461424 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5e63f99c-4ef1-4fb1-81fa-fc4bcb2c2ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:12.523582 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1391/2790] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:32:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:12.535865 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712f4c3-59f7-441f-9e8a-55c1aeb2b104 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:12.537607 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-4712f4c3-59f7-441f-9e8a-55c1aeb2b104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:12.556677 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:12.618106 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712f4c3-59f7-441f-9e8a-55c1aeb2b104 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:12.621756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591163d-0857-4857-b97f-8a6bf8f222b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:12.622382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591163d-0857-4857-b97f-8a6bf8f222b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:12.622863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591163d-0857-4857-b97f-8a6bf8f222b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:12.623175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591163d-0857-4857-b97f-8a6bf8f222b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:12.631908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591163d-0857-4857-b97f-8a6bf8f222b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:12.633343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591163d-0857-4857-b97f-8a6bf8f222b7 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:12.641972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1400/2791] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:22.526286 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:22.697878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=29 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:23.561330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f99c-4ef1-4fb1-81fa-fc4bcb2c2ec3 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:23.626145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f99c-4ef1-4fb1-81fa-fc4bcb2c2ec3 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:23.630693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:23.631330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:23.631611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:23.631953 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] PATH_INFO: `/v3/users/0d3efa570f0142d1963bacbc00971a62` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:23.633125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3efa570f0142d1963bacbc00971a62)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:23.635238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:23.819439 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3efa570f0142d1963bacbc00971a62'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:23.826918 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3efa570f0142d1963bacbc00971a62'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:23.914733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3efa570f0142d1963bacbc00971a62'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:23.915371 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3efa570f0142d1963bacbc00971a62'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:23.916016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3efa570f0142d1963bacbc00971a62'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:23.935732 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1392/2792] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:32:23 2022] DELETE /identity/v3/users/0d3efa570f0142d1963bacbc00971a62 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:23.940389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece7b15-c377-4f1e-bb97-1e03395d90fd None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:23.940909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece7b15-c377-4f1e-bb97-1e03395d90fd None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:23.941162 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece7b15-c377-4f1e-bb97-1e03395d90fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:23.960908 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2ece7b15-c377-4f1e-bb97-1e03395d90fd None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:23.962137 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2ece7b15-c377-4f1e-bb97-1e03395d90fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:24.021257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1401/2793] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:24.031392 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:24.032873 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:24.101729 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae4de4-1fe5-41c9-ab93-56780d343b78 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:24.107818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aad478-3894-4b85-9151-05d396a949eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:24.107818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aad478-3894-4b85-9151-05d396a949eb service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:24.107818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aad478-3894-4b85-9151-05d396a949eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:24.107818 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aad478-3894-4b85-9151-05d396a949eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:24.116611 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aad478-3894-4b85-9151-05d396a949eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:24.118450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aad478-3894-4b85-9151-05d396a949eb service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:24.130856 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1393/2794] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:24.651081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece7b15-c377-4f1e-bb97-1e03395d90fd None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:24.660995 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2ece7b15-c377-4f1e-bb97-1e03395d90fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:24.675877 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece7b15-c377-4f1e-bb97-1e03395d90fd None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:24.681265 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:24.681936 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:24.682213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:24.683124 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] PATH_INFO: `/v3/projects/4f9f4a213d9c4e35b0fdd385c0aaf339` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:24.684597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f9f4a213d9c4e35b0fdd385c0aaf339)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:24.686625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:24.786187 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f9f4a213d9c4e35b0fdd385c0aaf339', 'request_id': 'req-44937725-2694-457d-8bee-e45930bad912'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:24.798111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f9f4a213d9c4e35b0fdd385c0aaf339', 'request_id': 'req-44937725-2694-457d-8bee-e45930bad912'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:24.806716 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f9f4a213d9c4e35b0fdd385c0aaf339', 'request_id': 'req-44937725-2694-457d-8bee-e45930bad912'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:24.869497 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] The token cache is being invalidate because project 4f9f4a213d9c4e35b0fdd385c0aaf339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:32:24.869730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:24.870468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-44937725-2694-457d-8bee-e45930bad912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:24.872120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1402/2795] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:32:24 2022] DELETE /identity/v3/projects/4f9f4a213d9c4e35b0fdd385c0aaf339 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:29.262702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aad478-3894-4b85-9151-05d396a949eb service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:29.264295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e2aad478-3894-4b85-9151-05d396a949eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:29.275630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aad478-3894-4b85-9151-05d396a949eb service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:29.283599 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:29.324314 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55b0ad-661b-47e6-9949-c2a57f4332a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:29.324930 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55b0ad-661b-47e6-9949-c2a57f4332a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:29.325208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55b0ad-661b-47e6-9949-c2a57f4332a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:29.325479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55b0ad-661b-47e6-9949-c2a57f4332a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:29.327219 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-db55b0ad-661b-47e6-9949-c2a57f4332a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:29.408659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55b0ad-661b-47e6-9949-c2a57f4332a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:29.410063 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55b0ad-661b-47e6-9949-c2a57f4332a1 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:29.431002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1394/2796] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 01:32:34.865352 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:35.950692 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389cb5-fe0d-417b-a3b5-49570f5f18ec None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:35.951228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389cb5-fe0d-417b-a3b5-49570f5f18ec None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:35.951228 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab389cb5-fe0d-417b-a3b5-49570f5f18ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:35.973703 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-ab389cb5-fe0d-417b-a3b5-49570f5f18ec None None] MFA Rules not processed for user `f05648c4a067490693bd554f458c6a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:32:35.974925 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ab389cb5-fe0d-417b-a3b5-49570f5f18ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:36.030694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1403/2797] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:32:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:32:36.743475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55b0ad-661b-47e6-9949-c2a57f4332a1 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:36.744993 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-db55b0ad-661b-47e6-9949-c2a57f4332a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:36.755598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55b0ad-661b-47e6-9949-c2a57f4332a1 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:36.763300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc45159-d4a9-46ec-9ac0-77c1c8a38b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:36.764598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc45159-d4a9-46ec-9ac0-77c1c8a38b01 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:36.765343 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc45159-d4a9-46ec-9ac0-77c1c8a38b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:36.765747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc45159-d4a9-46ec-9ac0-77c1c8a38b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:36.768723 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-2cc45159-d4a9-46ec-9ac0-77c1c8a38b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:36.785963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc45159-d4a9-46ec-9ac0-77c1c8a38b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:36.787456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc45159-d4a9-46ec-9ac0-77c1c8a38b01 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:36.800198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1395/2798] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:41.018385 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab389cb5-fe0d-417b-a3b5-49570f5f18ec None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.019886 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-ab389cb5-fe0d-417b-a3b5-49570f5f18ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:41.091345 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab389cb5-fe0d-417b-a3b5-49570f5f18ec None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.096139 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.096806 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.097087 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.097361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] PATH_INFO: `/v3/users/165812ae8a304c1a94bc62dd84144d3c` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.098673 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=165812ae8a304c1a94bc62dd84144d3c)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.100616 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.282077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '165812ae8a304c1a94bc62dd84144d3c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:41.289538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '165812ae8a304c1a94bc62dd84144d3c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:41.347377 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '165812ae8a304c1a94bc62dd84144d3c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:41.348072 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '165812ae8a304c1a94bc62dd84144d3c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:41.362542 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '165812ae8a304c1a94bc62dd84144d3c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:41.369358 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1404/2799] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:32:41 2022] DELETE /identity/v3/users/165812ae8a304c1a94bc62dd84144d3c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:41.579550 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc45159-d4a9-46ec-9ac0-77c1c8a38b01 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:41.594835 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc45159-d4a9-46ec-9ac0-77c1c8a38b01 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:41.653572 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:41.654214 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:41.654491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:41.654765 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] PATH_INFO: `/v3/projects/10e81968971c4c3fa793ca081d168209` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:41.655942 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e81968971c4c3fa793ca081d168209)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:41.658008 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:41.680532 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=100 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:41.760831 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e81968971c4c3fa793ca081d168209', 'request_id': 'req-f914591e-0155-454f-8dc1-b3125496652e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:41.766913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e81968971c4c3fa793ca081d168209', 'request_id': 'req-f914591e-0155-454f-8dc1-b3125496652e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:41.850282 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e81968971c4c3fa793ca081d168209', 'request_id': 'req-f914591e-0155-454f-8dc1-b3125496652e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:41.861907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] The token cache is being invalidate because project 10e81968971c4c3fa793ca081d168209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80524) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:32:41.862198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:41.862816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:32:41.864364 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1396/2800] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:32:41 2022] DELETE /identity/v3/projects/10e81968971c4c3fa793ca081d168209 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:32:50.970519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:32:50.971999 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:51.003062 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:32:51.048512 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85675d09-84b5-4507-8a03-261c9bb99764 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:32:51.114868 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74d704-b5a6-446f-bbef-9a5a91a72f1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:32:51.115421 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74d704-b5a6-446f-bbef-9a5a91a72f1e service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:32:51.115706 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74d704-b5a6-446f-bbef-9a5a91a72f1e service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:32:51.116031 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74d704-b5a6-446f-bbef-9a5a91a72f1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:32:51.117861 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6e74d704-b5a6-446f-bbef-9a5a91a72f1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:32:51.130049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74d704-b5a6-446f-bbef-9a5a91a72f1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:32:51.131635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74d704-b5a6-446f-bbef-9a5a91a72f1e service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:32:51.142881 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1405/2801] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:32:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:32:51.858751 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:01.278378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=51 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:01.350125 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:01.352432 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:01.417077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914591e-0155-454f-8dc1-b3125496652e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:01.421208 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cd1b3-004e-479a-b9e0-d1d9c1e706ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:01.421847 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cd1b3-004e-479a-b9e0-d1d9c1e706ec service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:01.422280 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cd1b3-004e-479a-b9e0-d1d9c1e706ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:01.422565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cd1b3-004e-479a-b9e0-d1d9c1e706ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:01.424455 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-173cd1b3-004e-479a-b9e0-d1d9c1e706ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:01.435631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173cd1b3-004e-479a-b9e0-d1d9c1e706ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:01.437210 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173cd1b3-004e-479a-b9e0-d1d9c1e706ec service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:01.446902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1397/2802] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:03.982923 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74d704-b5a6-446f-bbef-9a5a91a72f1e service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:03.990132 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6e74d704-b5a6-446f-bbef-9a5a91a72f1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:03.999578 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74d704-b5a6-446f-bbef-9a5a91a72f1e service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:04.005714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6ac08-d0ca-4a33-b8eb-0f1494c87e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:04.006661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6ac08-d0ca-4a33-b8eb-0f1494c87e70 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:04.006661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6ac08-d0ca-4a33-b8eb-0f1494c87e70 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:04.006942 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6ac08-d0ca-4a33-b8eb-0f1494c87e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:04.009182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6bc6ac08-d0ca-4a33-b8eb-0f1494c87e70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:04.022348 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6ac08-d0ca-4a33-b8eb-0f1494c87e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:04.024704 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6ac08-d0ca-4a33-b8eb-0f1494c87e70 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:04.035675 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1406/2803] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:11.442119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=26 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:14.033111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=6 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:16.854030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cd1b3-004e-479a-b9e0-d1d9c1e706ec service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:16.855565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-173cd1b3-004e-479a-b9e0-d1d9c1e706ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:16.866222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cd1b3-004e-479a-b9e0-d1d9c1e706ec service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:16.869856 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa80c4-7f60-439d-abe8-bccff410c175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:16.870461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa80c4-7f60-439d-abe8-bccff410c175 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:16.870743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa80c4-7f60-439d-abe8-bccff410c175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:16.871025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa80c4-7f60-439d-abe8-bccff410c175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:16.872733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c6fa80c4-7f60-439d-abe8-bccff410c175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:16.935270 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa80c4-7f60-439d-abe8-bccff410c175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:16.936731 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa80c4-7f60-439d-abe8-bccff410c175 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:16.946763 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1398/2804] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:20.235577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6ac08-d0ca-4a33-b8eb-0f1494c87e70 service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:20.237350 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-6bc6ac08-d0ca-4a33-b8eb-0f1494c87e70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:20.308960 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6ac08-d0ca-4a33-b8eb-0f1494c87e70 service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:20.313809 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dad1ec-8b2c-4fd7-bd28-c8dacd751443 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:20.319883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dad1ec-8b2c-4fd7-bd28-c8dacd751443 service placement] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:20.319883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dad1ec-8b2c-4fd7-bd28-c8dacd751443 service placement] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:20.319883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dad1ec-8b2c-4fd7-bd28-c8dacd751443 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:20.319883 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-13dad1ec-8b2c-4fd7-bd28-c8dacd751443 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:20.327807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dad1ec-8b2c-4fd7-bd28-c8dacd751443 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:20.329424 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dad1ec-8b2c-4fd7-bd28-c8dacd751443 service placement] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:20.339077 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1407/2805] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:24.000603 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa80c4-7f60-439d-abe8-bccff410c175 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.002092 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c6fa80c4-7f60-439d-abe8-bccff410c175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:24.015527 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa80c4-7f60-439d-abe8-bccff410c175 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.020594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.021187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.021480 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.021762 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] PATH_INFO: `/v3/users/42047fe93eb24adbbc24844a75ed21f2` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.022943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42047fe93eb24adbbc24844a75ed21f2)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.024984 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.199725 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42047fe93eb24adbbc24844a75ed21f2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.206610 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42047fe93eb24adbbc24844a75ed21f2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.279012 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42047fe93eb24adbbc24844a75ed21f2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.279558 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42047fe93eb24adbbc24844a75ed21f2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.280187 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42047fe93eb24adbbc24844a75ed21f2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.298450 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1399/2806] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:23 2022] DELETE /identity/v3/users/42047fe93eb24adbbc24844a75ed21f2 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:24.520661 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dad1ec-8b2c-4fd7-bd28-c8dacd751443 service placement] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.532168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dad1ec-8b2c-4fd7-bd28-c8dacd751443 service placement] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.589069 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.589719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.590005 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.590288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] PATH_INFO: `/v3/projects/aeee08ad0a654fc1839cc533fc804479` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.591460 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeee08ad0a654fc1839cc533fc804479)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.593395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.685147 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeee08ad0a654fc1839cc533fc804479', 'request_id': 'req-b30374f0-aaab-4710-aaf7-2f881a12ba78'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.691339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeee08ad0a654fc1839cc533fc804479', 'request_id': 'req-b30374f0-aaab-4710-aaf7-2f881a12ba78'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.696854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeee08ad0a654fc1839cc533fc804479', 'request_id': 'req-b30374f0-aaab-4710-aaf7-2f881a12ba78'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.770052 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] The token cache is being invalidate because project aeee08ad0a654fc1839cc533fc804479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:24.770344 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.770972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:24.772630 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1408/2807] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:24 2022] DELETE /identity/v3/projects/aeee08ad0a654fc1839cc533fc804479 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:24.776686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:24.778255 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:24.792138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348998c-2358-4d79-8936-eaeee3a759b3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:24.847098 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:24.847733 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:24.848110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:24.848386 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] PATH_INFO: `/v3/users/c67f5a5e953c42e8a33f5b92a006e479` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:24.849534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c67f5a5e953c42e8a33f5b92a006e479)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:24.852027 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:24.857344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:25.024458 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c67f5a5e953c42e8a33f5b92a006e479'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.031238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c67f5a5e953c42e8a33f5b92a006e479'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.107862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c67f5a5e953c42e8a33f5b92a006e479'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.108410 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c67f5a5e953c42e8a33f5b92a006e479'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.109110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-c90a01eb-6e4e-4bf1-85e6-b9c75447530a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c67f5a5e953c42e8a33f5b92a006e479'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.131308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1400/2808] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:33:24 2022] DELETE /identity/v3/users/c67f5a5e953c42e8a33f5b92a006e479 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:25.349266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:25.350711 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:25.364168 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30374f0-aaab-4710-aaf7-2f881a12ba78 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:25.382018 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:25.422499 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:25.423186 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:25.423519 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:25.423830 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] PATH_INFO: `/v3/projects/c7cf8e9bf49647a2884bddbea672f616` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:25.424992 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7cf8e9bf49647a2884bddbea672f616)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:25.427022 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:25.525315 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7cf8e9bf49647a2884bddbea672f616', 'request_id': 'req-d96c9640-3b6e-459f-99bd-32bfa9228e3b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.531565 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7cf8e9bf49647a2884bddbea672f616', 'request_id': 'req-d96c9640-3b6e-459f-99bd-32bfa9228e3b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.537807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7cf8e9bf49647a2884bddbea672f616', 'request_id': 'req-d96c9640-3b6e-459f-99bd-32bfa9228e3b'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.632199 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] The token cache is being invalidate because project c7cf8e9bf49647a2884bddbea672f616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:33:25.632400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.633024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:33:25.634553 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1409/2809] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:33:25 2022] DELETE /identity/v3/projects/c7cf8e9bf49647a2884bddbea672f616 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:33:35.125281 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:35.628723 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:44.989334 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372231c-ceb9-4d66-ad1d-542b8bf1f40e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:44.989816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372231c-ceb9-4d66-ad1d-542b8bf1f40e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:44.989933 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372231c-ceb9-4d66-ad1d-542b8bf1f40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:45.009702 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-3372231c-ceb9-4d66-ad1d-542b8bf1f40e None None] MFA Rules not processed for user `f05648c4a067490693bd554f458c6a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:33:45.015145 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3372231c-ceb9-4d66-ad1d-542b8bf1f40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:45.063547 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1401/2810] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:33:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:33:45.357623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:45.359033 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:45.414448 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c9640-3b6e-459f-99bd-32bfa9228e3b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:45.418683 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fa942-8417-4d92-b6c1-222c7bbcb9a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:45.419288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fa942-8417-4d92-b6c1-222c7bbcb9a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:45.419567 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fa942-8417-4d92-b6c1-222c7bbcb9a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:45.419874 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fa942-8417-4d92-b6c1-222c7bbcb9a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:45.421574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d80fa942-8417-4d92-b6c1-222c7bbcb9a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:45.430928 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fa942-8417-4d92-b6c1-222c7bbcb9a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:45.432405 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fa942-8417-4d92-b6c1-222c7bbcb9a4 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:45.441213 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1410/2811] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:46.838289 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372231c-ceb9-4d66-ad1d-542b8bf1f40e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:46.839693 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-3372231c-ceb9-4d66-ad1d-542b8bf1f40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:46.909293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372231c-ceb9-4d66-ad1d-542b8bf1f40e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:46.914367 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79d5a1-166d-4db4-b7d8-6bdfe27ab45e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:46.915576 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d5a1-166d-4db4-b7d8-6bdfe27ab45e service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:46.915894 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d5a1-166d-4db4-b7d8-6bdfe27ab45e service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:46.916176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d5a1-166d-4db4-b7d8-6bdfe27ab45e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:46.917961 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7d79d5a1-166d-4db4-b7d8-6bdfe27ab45e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:46.930006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79d5a1-166d-4db4-b7d8-6bdfe27ab45e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:46.931643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79d5a1-166d-4db4-b7d8-6bdfe27ab45e service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:46.941722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1402/2812] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:33:55.437126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=26 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:56.937340 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=57 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:33:59.063190 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fa942-8417-4d92-b6c1-222c7bbcb9a4 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:33:59.064873 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d80fa942-8417-4d92-b6c1-222c7bbcb9a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:59.139631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fa942-8417-4d92-b6c1-222c7bbcb9a4 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:33:59.145177 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85381739-86aa-4c73-a878-985c0fb5e8a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db1b205d165543d0a3a69a2db6af5d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:33:59.145841 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85381739-86aa-4c73-a878-985c0fb5e8a7 service nova] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:33:59.146118 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85381739-86aa-4c73-a878-985c0fb5e8a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:33:59.146388 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85381739-86aa-4c73-a878-985c0fb5e8a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:33:59.148451 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-85381739-86aa-4c73-a878-985c0fb5e8a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:33:59.161482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85381739-86aa-4c73-a878-985c0fb5e8a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:33:59.163057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85381739-86aa-4c73-a878-985c0fb5e8a7 service nova] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:33:59.174544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1411/2813] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:33:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:09.169800 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=34 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:46.853799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79d5a1-166d-4db4-b7d8-6bdfe27ab45e service placement] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:46.855216 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-7d79d5a1-166d-4db4-b7d8-6bdfe27ab45e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:46.910379 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79d5a1-166d-4db4-b7d8-6bdfe27ab45e service placement] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:46.913980 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fa7ae-bbad-46dc-9ea3-649d5a5eeb9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:46.914587 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fa7ae-bbad-46dc-9ea3-649d5a5eeb9d service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:46.914867 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fa7ae-bbad-46dc-9ea3-649d5a5eeb9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:46.915138 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fa7ae-bbad-46dc-9ea3-649d5a5eeb9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:46.916870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-090fa7ae-bbad-46dc-9ea3-649d5a5eeb9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:46.928847 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fa7ae-bbad-46dc-9ea3-649d5a5eeb9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:46.930509 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fa7ae-bbad-46dc-9ea3-649d5a5eeb9d service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:46.941028 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1403/2814] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:34:56.935328 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=26 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:34:59.522386 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85381739-86aa-4c73-a878-985c0fb5e8a7 service nova] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:34:59.531844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85381739-86aa-4c73-a878-985c0fb5e8a7 service nova] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:34:59.535355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ae37-86a2-46d5-82c6-b4854f6a3464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:34:59.536000 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ae37-86a2-46d5-82c6-b4854f6a3464 service neutron] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:34:59.536289 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ae37-86a2-46d5-82c6-b4854f6a3464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:34:59.536569 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ae37-86a2-46d5-82c6-b4854f6a3464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:34:59.538341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-b1c1ae37-86a2-46d5-82c6-b4854f6a3464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:34:59.602517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1ae37-86a2-46d5-82c6-b4854f6a3464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:34:59.604036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1ae37-86a2-46d5-82c6-b4854f6a3464 service neutron] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:34:59.613789 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1412/2815] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:02.839181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fa7ae-bbad-46dc-9ea3-649d5a5eeb9d service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:02.840662 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-090fa7ae-bbad-46dc-9ea3-649d5a5eeb9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:02.855319 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fa7ae-bbad-46dc-9ea3-649d5a5eeb9d service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:02.859907 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:02.861061 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:02.861347 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:02.861625 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] PATH_INFO: `/v3/users/f05648c4a067490693bd554f458c6a1b` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:02.862806 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05648c4a067490693bd554f458c6a1b)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:02.865034 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:03.005853 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05648c4a067490693bd554f458c6a1b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:03.012801 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05648c4a067490693bd554f458c6a1b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:03.066301 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05648c4a067490693bd554f458c6a1b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:03.066921 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05648c4a067490693bd554f458c6a1b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:03.067530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-f64681c3-d1b8-4ef3-a685-296c47cb4393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05648c4a067490693bd554f458c6a1b'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:03.085006 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1404/2816] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:02 2022] DELETE /identity/v3/users/f05648c4a067490693bd554f458c6a1b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:03.320101 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ae37-86a2-46d5-82c6-b4854f6a3464 service neutron] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:03.330341 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ae37-86a2-46d5-82c6-b4854f6a3464 service neutron] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:03.393666 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:03.394304 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:03.394614 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:03.394887 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] PATH_INFO: `/v3/projects/2e1f3b871e254774a05408b816015827` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:03.396063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1f3b871e254774a05408b816015827)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:03.397941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:03.495544 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1f3b871e254774a05408b816015827', 'request_id': 'req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:03.501730 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1f3b871e254774a05408b816015827', 'request_id': 'req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:03.507176 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1f3b871e254774a05408b816015827', 'request_id': 'req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:03.545370 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] The token cache is being invalidate because project 2e1f3b871e254774a05408b816015827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:03.545662 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:03.546313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ae58f3fc-a800-4a8f-9e16-1c04ef6eb78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:03.548623 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1413/2817] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:03 2022] DELETE /identity/v3/projects/2e1f3b871e254774a05408b816015827 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:03.553641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d691237-a3a9-4e1d-bbe1-126030652c8d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:03.560043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d691237-a3a9-4e1d-bbe1-126030652c8d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:03.560043 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d691237-a3a9-4e1d-bbe1-126030652c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:03.574331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-5d691237-a3a9-4e1d-bbe1-126030652c8d None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:03.575475 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-5d691237-a3a9-4e1d-bbe1-126030652c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:03.637795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1405/2818] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:03.647631 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fff746-907c-40c7-928a-e8fa6cf65841 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:03.648079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fff746-907c-40c7-928a-e8fa6cf65841 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:03.648365 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fff746-907c-40c7-928a-e8fa6cf65841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:03.668389 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-31fff746-907c-40c7-928a-e8fa6cf65841 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:03.669638 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-31fff746-907c-40c7-928a-e8fa6cf65841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:03.677406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1414/2819] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:03.681548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d691237-a3a9-4e1d-bbe1-126030652c8d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:03.691093 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d691237-a3a9-4e1d-bbe1-126030652c8d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:03.695630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d0509-f67c-41da-b6d9-dbc208bf16aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:03.696268 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0509-f67c-41da-b6d9-dbc208bf16aa admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:03.696553 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0509-f67c-41da-b6d9-dbc208bf16aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:03.696833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0509-f67c-41da-b6d9-dbc208bf16aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:03.697546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d0509-f67c-41da-b6d9-dbc208bf16aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:35:03.697852 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d0509-f67c-41da-b6d9-dbc208bf16aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:03.699449 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d0509-f67c-41da-b6d9-dbc208bf16aa admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:03.711059 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1406/2820] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:35:03.715170 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fff746-907c-40c7-928a-e8fa6cf65841 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:03.724720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fff746-907c-40c7-928a-e8fa6cf65841 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:03.729269 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7aacb-8e0b-4af4-8ea8-34bdc3a61d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:03.729982 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7aacb-8e0b-4af4-8ea8-34bdc3a61d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:03.730262 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7aacb-8e0b-4af4-8ea8-34bdc3a61d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:03.730532 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7aacb-8e0b-4af4-8ea8-34bdc3a61d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:03.731274 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7aacb-8e0b-4af4-8ea8-34bdc3a61d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:03.733181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7aacb-8e0b-4af4-8ea8-34bdc3a61d81 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:03.748414 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:03.810275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1415/2821] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:03 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:35:03.814505 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d0509-f67c-41da-b6d9-dbc208bf16aa admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:03.824503 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d0509-f67c-41da-b6d9-dbc208bf16aa admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:03.886862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6405dd7-a292-402c-921f-87a16e1a225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:03.887498 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405dd7-a292-402c-921f-87a16e1a225c admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:03.887811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405dd7-a292-402c-921f-87a16e1a225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:03.888084 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6405dd7-a292-402c-921f-87a16e1a225c admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:03.888743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6405dd7-a292-402c-921f-87a16e1a225c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:03.890671 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6405dd7-a292-402c-921f-87a16e1a225c admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:03.899215 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:03.963552 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1407/2822] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:03 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:03.967879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7aacb-8e0b-4af4-8ea8-34bdc3a61d81 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:03.978775 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7aacb-8e0b-4af4-8ea8-34bdc3a61d81 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:03.983375 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c9e59-9655-4dde-9162-cd034522abb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:03.984012 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c9e59-9655-4dde-9162-cd034522abb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:03.984297 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c9e59-9655-4dde-9162-cd034522abb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:03.984576 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c9e59-9655-4dde-9162-cd034522abb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:03.985273 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c9e59-9655-4dde-9162-cd034522abb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:03.986884 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c9e59-9655-4dde-9162-cd034522abb3 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:03.996940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1416/2823] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:04.000946 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6405dd7-a292-402c-921f-87a16e1a225c admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:04.011341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6405dd7-a292-402c-921f-87a16e1a225c admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:04.016013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e5b05-27eb-4250-a1e6-df0ed3f8dc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:04.016621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e5b05-27eb-4250-a1e6-df0ed3f8dc5b admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.017494 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e5b05-27eb-4250-a1e6-df0ed3f8dc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.017772 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e5b05-27eb-4250-a1e6-df0ed3f8dc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.018463 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e5b05-27eb-4250-a1e6-df0ed3f8dc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:04.020191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e5b05-27eb-4250-a1e6-df0ed3f8dc5b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:04.031180 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1408/2824] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:04.035211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c9e59-9655-4dde-9162-cd034522abb3 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:04.045004 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c9e59-9655-4dde-9162-cd034522abb3 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:04.049556 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e1cb9-fb9b-4faf-896f-324f391e7f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:04.050201 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e1cb9-fb9b-4faf-896f-324f391e7f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.050484 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e1cb9-fb9b-4faf-896f-324f391e7f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.050763 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e1cb9-fb9b-4faf-896f-324f391e7f2e admin admin] PATH_INFO: `/v3/projects/386879fb35484be99fdd58a0234b4bf9/users/e094fef63c4c43d6a63a1cf8ce7f037e/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.065367 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e1cb9-fb9b-4faf-896f-324f391e7f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=386879fb35484be99fdd58a0234b4bf9, user_id=e094fef63c4c43d6a63a1cf8ce7f037e, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:04.068036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e1cb9-fb9b-4faf-896f-324f391e7f2e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:04.186762 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1417/2825] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:04 2022] PUT /identity/v3/projects/386879fb35484be99fdd58a0234b4bf9/users/e094fef63c4c43d6a63a1cf8ce7f037e/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:35:04.197521 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97fee-e35a-4f8d-9bf1-1f8b3b15ee1d None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.197816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97fee-e35a-4f8d-9bf1-1f8b3b15ee1d None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.198101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97fee-e35a-4f8d-9bf1-1f8b3b15ee1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.225748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-55e97fee-e35a-4f8d-9bf1-1f8b3b15ee1d None None] MFA Rules not processed for user `e094fef63c4c43d6a63a1cf8ce7f037e`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:04.226959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-55e97fee-e35a-4f8d-9bf1-1f8b3b15ee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:04.288914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1409/2826] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:04.293382 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e1cb9-fb9b-4faf-896f-324f391e7f2e admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:04.353922 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e1cb9-fb9b-4faf-896f-324f391e7f2e admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:04.358470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7ee4-159a-4794-ab16-e700d83408aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:04.359076 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7ee4-159a-4794-ab16-e700d83408aa admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.359355 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7ee4-159a-4794-ab16-e700d83408aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.359628 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a7ee4-159a-4794-ab16-e700d83408aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.360843 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a7ee4-159a-4794-ab16-e700d83408aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:04.362754 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8a7ee4-159a-4794-ab16-e700d83408aa admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:04.479007 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1418/2827] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:04 2022] POST /identity/v3/projects => generated 379 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:04.483199 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e97fee-e35a-4f8d-9bf1-1f8b3b15ee1d None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:04.494787 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e97fee-e35a-4f8d-9bf1-1f8b3b15ee1d None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:04.549889 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1dbdd1-7df2-4c53-9da8-903f2d01e85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:04.550517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dbdd1-7df2-4c53-9da8-903f2d01e85a admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.550799 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dbdd1-7df2-4c53-9da8-903f2d01e85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.551074 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1dbdd1-7df2-4c53-9da8-903f2d01e85a admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.551797 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1dbdd1-7df2-4c53-9da8-903f2d01e85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:04.553757 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1dbdd1-7df2-4c53-9da8-903f2d01e85a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:04.623104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1410/2828] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:04 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:04.625411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7ee4-159a-4794-ab16-e700d83408aa admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:04.636301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a7ee4-159a-4794-ab16-e700d83408aa admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:04.640863 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c996db1-cac7-46b7-8bf9-1e64642f7649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:04.641470 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c996db1-cac7-46b7-8bf9-1e64642f7649 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.641756 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c996db1-cac7-46b7-8bf9-1e64642f7649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.642037 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c996db1-cac7-46b7-8bf9-1e64642f7649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.642709 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c996db1-cac7-46b7-8bf9-1e64642f7649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:04.644323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c996db1-cac7-46b7-8bf9-1e64642f7649 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:04.655740 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1419/2829] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:04.660250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1dbdd1-7df2-4c53-9da8-903f2d01e85a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:04.670526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1dbdd1-7df2-4c53-9da8-903f2d01e85a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:04.675657 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85488a52-b21b-4227-8d4a-7256e8caaff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:04.676356 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85488a52-b21b-4227-8d4a-7256e8caaff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.676641 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85488a52-b21b-4227-8d4a-7256e8caaff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.676923 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85488a52-b21b-4227-8d4a-7256e8caaff0 admin admin] PATH_INFO: `/v3/projects/0bd5c1b5fc7b4b85bc764a2b58a78476/users/581446b08a1845249878c8b65b72c977/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.692127 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85488a52-b21b-4227-8d4a-7256e8caaff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd5c1b5fc7b4b85bc764a2b58a78476, user_id=581446b08a1845249878c8b65b72c977, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:04.694901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85488a52-b21b-4227-8d4a-7256e8caaff0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:04.763181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1411/2830] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:04 2022] PUT /identity/v3/projects/0bd5c1b5fc7b4b85bc764a2b58a78476/users/581446b08a1845249878c8b65b72c977/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:04.767060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c996db1-cac7-46b7-8bf9-1e64642f7649 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:04.776906 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c996db1-cac7-46b7-8bf9-1e64642f7649 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:04.833094 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a0530-e465-4b11-b789-4e64dd6b9659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:04.833709 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a0530-e465-4b11-b789-4e64dd6b9659 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.833991 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a0530-e465-4b11-b789-4e64dd6b9659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.834266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a0530-e465-4b11-b789-4e64dd6b9659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.834956 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a0530-e465-4b11-b789-4e64dd6b9659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:04.836636 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a0530-e465-4b11-b789-4e64dd6b9659 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:04.846668 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1420/2831] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:04.850959 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85488a52-b21b-4227-8d4a-7256e8caaff0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:04.861774 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85488a52-b21b-4227-8d4a-7256e8caaff0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:04.866639 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6462a-9670-4037-bd16-115d158aa4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:04.867298 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6462a-9670-4037-bd16-115d158aa4b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.867577 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6462a-9670-4037-bd16-115d158aa4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.867902 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6462a-9670-4037-bd16-115d158aa4b1 admin admin] PATH_INFO: `/v3/projects/0bd5c1b5fc7b4b85bc764a2b58a78476/users/581446b08a1845249878c8b65b72c977/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.869816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6462a-9670-4037-bd16-115d158aa4b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd5c1b5fc7b4b85bc764a2b58a78476, user_id=581446b08a1845249878c8b65b72c977, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:04.872539 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6462a-9670-4037-bd16-115d158aa4b1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:04.882700 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1412/2832] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:04 2022] PUT /identity/v3/projects/0bd5c1b5fc7b4b85bc764a2b58a78476/users/581446b08a1845249878c8b65b72c977/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:04.892837 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f530a64-0aa7-44a6-b652-8ac3f0659065 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:04.893128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f530a64-0aa7-44a6-b652-8ac3f0659065 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:04.893404 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f530a64-0aa7-44a6-b652-8ac3f0659065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:04.920924 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-2f530a64-0aa7-44a6-b652-8ac3f0659065 None None] MFA Rules not processed for user `581446b08a1845249878c8b65b72c977`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:04.922142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-2f530a64-0aa7-44a6-b652-8ac3f0659065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:04.931728 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:04.990767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1421/2833] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:05.002151 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6462a-9670-4037-bd16-115d158aa4b1 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:05.003898 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-74b6462a-9670-4037-bd16-115d158aa4b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:05.025355 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:05.060510 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6462a-9670-4037-bd16-115d158aa4b1 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:05.064142 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd212efd-43f8-4728-aa3c-5768ee336386 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:05.064764 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd212efd-43f8-4728-aa3c-5768ee336386 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.065236 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd212efd-43f8-4728-aa3c-5768ee336386 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.065517 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd212efd-43f8-4728-aa3c-5768ee336386 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.079015 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd212efd-43f8-4728-aa3c-5768ee336386 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:05.080524 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd212efd-43f8-4728-aa3c-5768ee336386 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:05.090548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1413/2834] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:05.308014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f530a64-0aa7-44a6-b652-8ac3f0659065 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:05.317593 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f530a64-0aa7-44a6-b652-8ac3f0659065 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:05.321109 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfffef-bbd0-4493-8d92-aeff82b4e2cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:05.321720 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfffef-bbd0-4493-8d92-aeff82b4e2cc service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:05.322023 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfffef-bbd0-4493-8d92-aeff82b4e2cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:05.322272 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfffef-bbd0-4493-8d92-aeff82b4e2cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:05.378024 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfffef-bbd0-4493-8d92-aeff82b4e2cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:05.379462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfffef-bbd0-4493-8d92-aeff82b4e2cc service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:05.388715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1422/2835] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:15.086717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=17 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:15.384394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=52 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:15.572228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd212efd-43f8-4728-aa3c-5768ee336386 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:15.635263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd212efd-43f8-4728-aa3c-5768ee336386 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:15.639813 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:15.640459 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:15.640743 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:15.641016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] PATH_INFO: `/v3/users/e094fef63c4c43d6a63a1cf8ce7f037e` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:15.642159 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e094fef63c4c43d6a63a1cf8ce7f037e)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:15.644148 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:15.817593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e094fef63c4c43d6a63a1cf8ce7f037e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:15.824506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e094fef63c4c43d6a63a1cf8ce7f037e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:15.871972 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e094fef63c4c43d6a63a1cf8ce7f037e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:15.872583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e094fef63c4c43d6a63a1cf8ce7f037e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:15.873192 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e094fef63c4c43d6a63a1cf8ce7f037e'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:15.891385 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1414/2836] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:15 2022] DELETE /identity/v3/users/e094fef63c4c43d6a63a1cf8ce7f037e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:15.896468 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a48de5-c5bd-4b84-8f4c-d92c1e3fa9df None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:15.896752 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a48de5-c5bd-4b84-8f4c-d92c1e3fa9df None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:15.903043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a48de5-c5bd-4b84-8f4c-d92c1e3fa9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:15.916533 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-38a48de5-c5bd-4b84-8f4c-d92c1e3fa9df None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:15.917719 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-38a48de5-c5bd-4b84-8f4c-d92c1e3fa9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:15.974660 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1423/2837] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:15.984330 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:15.985704 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:16.041085 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597f267-8f46-4eca-9431-42106e8b4ea3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:16.044650 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2175a27-1ff4-4dd6-9aa8-ae4e1a7ca48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:16.045263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2175a27-1ff4-4dd6-9aa8-ae4e1a7ca48d service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:16.045538 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2175a27-1ff4-4dd6-9aa8-ae4e1a7ca48d service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:16.045812 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2175a27-1ff4-4dd6-9aa8-ae4e1a7ca48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:16.054554 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2175a27-1ff4-4dd6-9aa8-ae4e1a7ca48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:16.056000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2175a27-1ff4-4dd6-9aa8-ae4e1a7ca48d service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:16.065697 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1415/2838] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:16.714232 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a48de5-c5bd-4b84-8f4c-d92c1e3fa9df None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:16.715690 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-38a48de5-c5bd-4b84-8f4c-d92c1e3fa9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:16.732905 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a48de5-c5bd-4b84-8f4c-d92c1e3fa9df None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:16.738400 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:16.739334 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:16.739619 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:16.739933 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] PATH_INFO: `/v3/projects/386879fb35484be99fdd58a0234b4bf9` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:16.741104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=386879fb35484be99fdd58a0234b4bf9)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:16.743016 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:16.843397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '386879fb35484be99fdd58a0234b4bf9', 'request_id': 'req-d4883595-9515-4c25-acda-73a62906cc4f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.850149 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '386879fb35484be99fdd58a0234b4bf9', 'request_id': 'req-d4883595-9515-4c25-acda-73a62906cc4f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.855640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '386879fb35484be99fdd58a0234b4bf9', 'request_id': 'req-d4883595-9515-4c25-acda-73a62906cc4f'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.923313 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] The token cache is being invalidate because project 386879fb35484be99fdd58a0234b4bf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:16.923598 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.924846 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:16.926361 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1424/2839] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:16 2022] DELETE /identity/v3/projects/386879fb35484be99fdd58a0234b4bf9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:16.930322 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2175a27-1ff4-4dd6-9aa8-ae4e1a7ca48d service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:16.939722 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2175a27-1ff4-4dd6-9aa8-ae4e1a7ca48d service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:16.957135 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:16.995345 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:16.996016 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:16.996349 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:16.996817 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] PATH_INFO: `/v3/users/581446b08a1845249878c8b65b72c977` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:16.998391 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=581446b08a1845249878c8b65b72c977)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:17.000390 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:17.176821 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '581446b08a1845249878c8b65b72c977'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.183745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '581446b08a1845249878c8b65b72c977'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.248909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '581446b08a1845249878c8b65b72c977'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.249512 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '581446b08a1845249878c8b65b72c977'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.250106 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-7be0a32c-b9b2-4cdb-a41f-0accd4b813f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '581446b08a1845249878c8b65b72c977'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.266939 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1416/2840] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:16 2022] DELETE /identity/v3/users/581446b08a1845249878c8b65b72c977 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:17.645981 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:17.647439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:17.660343 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4883595-9515-4c25-acda-73a62906cc4f admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:17.716342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:17.716948 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.717196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.717469 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] PATH_INFO: `/v3/projects/0bd5c1b5fc7b4b85bc764a2b58a78476` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.719836 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd5c1b5fc7b4b85bc764a2b58a78476)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:17.720546 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:17.799309 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:17.814406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd5c1b5fc7b4b85bc764a2b58a78476', 'request_id': 'req-4684c9b9-5c64-4fdc-919d-6df75bba7095'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.821257 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd5c1b5fc7b4b85bc764a2b58a78476', 'request_id': 'req-4684c9b9-5c64-4fdc-919d-6df75bba7095'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.826694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd5c1b5fc7b4b85bc764a2b58a78476', 'request_id': 'req-4684c9b9-5c64-4fdc-919d-6df75bba7095'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.900423 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] The token cache is being invalidate because project 0bd5c1b5fc7b4b85bc764a2b58a78476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:17.900715 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.901329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-4684c9b9-5c64-4fdc-919d-6df75bba7095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:17.902892 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1425/2841] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:17 2022] DELETE /identity/v3/projects/0bd5c1b5fc7b4b85bc764a2b58a78476 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:17.909000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990f126-5abe-4a37-b524-fcef3ec2f11e None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.909293 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990f126-5abe-4a37-b524-fcef3ec2f11e None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.909578 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990f126-5abe-4a37-b524-fcef3ec2f11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:17.929708 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-f990f126-5abe-4a37-b524-fcef3ec2f11e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:17.930920 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f990f126-5abe-4a37-b524-fcef3ec2f11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:17.988262 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1417/2842] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:17.996428 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a0df7-1f2e-4dcd-bfac-3642e7736bb9 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:17.996714 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a0df7-1f2e-4dcd-bfac-3642e7736bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:17.996989 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a0df7-1f2e-4dcd-bfac-3642e7736bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.016573 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-689a0df7-1f2e-4dcd-bfac-3642e7736bb9 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:18.017785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-689a0df7-1f2e-4dcd-bfac-3642e7736bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:18.025839 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1426/2843] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:18.029833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990f126-5abe-4a37-b524-fcef3ec2f11e None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.039309 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990f126-5abe-4a37-b524-fcef3ec2f11e None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.043819 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261242c-9d3a-441b-8a2f-fba3c9a74ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.044427 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261242c-9d3a-441b-8a2f-fba3c9a74ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.044707 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261242c-9d3a-441b-8a2f-fba3c9a74ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.044982 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261242c-9d3a-441b-8a2f-fba3c9a74ac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.045668 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261242c-9d3a-441b-8a2f-fba3c9a74ac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:35:18.045977 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261242c-9d3a-441b-8a2f-fba3c9a74ac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.047571 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261242c-9d3a-441b-8a2f-fba3c9a74ac2 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.059185 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1418/2844] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:35:18.063314 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a0df7-1f2e-4dcd-bfac-3642e7736bb9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.073055 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a0df7-1f2e-4dcd-bfac-3642e7736bb9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.077686 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc7469-c597-499f-b76d-3d40f4165cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.078288 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc7469-c597-499f-b76d-3d40f4165cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.078574 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc7469-c597-499f-b76d-3d40f4165cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.078854 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc7469-c597-499f-b76d-3d40f4165cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.079518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc7469-c597-499f-b76d-3d40f4165cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.081457 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc7469-c597-499f-b76d-3d40f4165cb9 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.164872 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1427/2845] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:18 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:18.168932 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261242c-9d3a-441b-8a2f-fba3c9a74ac2 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.179506 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261242c-9d3a-441b-8a2f-fba3c9a74ac2 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.205039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:18.234795 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ba083-6b4c-4326-a1ec-2d3112c46b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.235415 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ba083-6b4c-4326-a1ec-2d3112c46b36 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.235694 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ba083-6b4c-4326-a1ec-2d3112c46b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.236004 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ba083-6b4c-4326-a1ec-2d3112c46b36 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.237260 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ba083-6b4c-4326-a1ec-2d3112c46b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.239832 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ba083-6b4c-4326-a1ec-2d3112c46b36 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.353032 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1419/2846] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:18 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:18.356938 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc7469-c597-499f-b76d-3d40f4165cb9 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.368064 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc7469-c597-499f-b76d-3d40f4165cb9 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.372656 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3e840-b74a-47ba-b84a-d57ad4e0db24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.373256 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3e840-b74a-47ba-b84a-d57ad4e0db24 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.373539 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3e840-b74a-47ba-b84a-d57ad4e0db24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.373814 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3e840-b74a-47ba-b84a-d57ad4e0db24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.374506 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3e840-b74a-47ba-b84a-d57ad4e0db24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.376125 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3e840-b74a-47ba-b84a-d57ad4e0db24 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.385821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1428/2847] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:18.389591 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ba083-6b4c-4326-a1ec-2d3112c46b36 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.400022 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ba083-6b4c-4326-a1ec-2d3112c46b36 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.404696 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4e94e-7299-4c66-b39d-70607d4ff973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.405290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4e94e-7299-4c66-b39d-70607d4ff973 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.405573 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4e94e-7299-4c66-b39d-70607d4ff973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.405850 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4e94e-7299-4c66-b39d-70607d4ff973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.406537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4e94e-7299-4c66-b39d-70607d4ff973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.408178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4e94e-7299-4c66-b39d-70607d4ff973 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.418313 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1420/2848] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:18.422200 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3e840-b74a-47ba-b84a-d57ad4e0db24 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.432524 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3e840-b74a-47ba-b84a-d57ad4e0db24 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.437082 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b171cf-bba8-4df7-8809-36ee4db2de10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.437727 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b171cf-bba8-4df7-8809-36ee4db2de10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.438010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b171cf-bba8-4df7-8809-36ee4db2de10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.438293 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b171cf-bba8-4df7-8809-36ee4db2de10 admin admin] PATH_INFO: `/v3/projects/7f0456227ea8478dbac663788e6b1dd9/users/fb3ed322b74346718df312e79dc2e7df/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.453477 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b171cf-bba8-4df7-8809-36ee4db2de10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0456227ea8478dbac663788e6b1dd9, user_id=fb3ed322b74346718df312e79dc2e7df, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.456184 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b171cf-bba8-4df7-8809-36ee4db2de10 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.540812 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1429/2849] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:18 2022] PUT /identity/v3/projects/7f0456227ea8478dbac663788e6b1dd9/users/fb3ed322b74346718df312e79dc2e7df/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:35:18.553903 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2c0a9-8386-4f76-bdeb-78e70a909429 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.554196 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2c0a9-8386-4f76-bdeb-78e70a909429 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.554478 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2c0a9-8386-4f76-bdeb-78e70a909429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.582948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-aed2c0a9-8386-4f76-bdeb-78e70a909429 None None] MFA Rules not processed for user `fb3ed322b74346718df312e79dc2e7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:18.584199 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-aed2c0a9-8386-4f76-bdeb-78e70a909429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:18.646769 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1421/2850] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:18.651197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b171cf-bba8-4df7-8809-36ee4db2de10 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.711411 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b171cf-bba8-4df7-8809-36ee4db2de10 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.716068 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bfbb3-7b21-454d-a6f5-e7f8dc7c854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.716685 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bfbb3-7b21-454d-a6f5-e7f8dc7c854a admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.717235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bfbb3-7b21-454d-a6f5-e7f8dc7c854a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.717235 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bfbb3-7b21-454d-a6f5-e7f8dc7c854a admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.717879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858bfbb3-7b21-454d-a6f5-e7f8dc7c854a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.719751 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858bfbb3-7b21-454d-a6f5-e7f8dc7c854a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.816267 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1430/2851] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:18 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:35:18.820344 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2c0a9-8386-4f76-bdeb-78e70a909429 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.831363 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2c0a9-8386-4f76-bdeb-78e70a909429 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.886119 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2051e4-fd0d-4d9e-bae1-afffd826821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:18.886741 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2051e4-fd0d-4d9e-bae1-afffd826821b admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:18.887021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2051e4-fd0d-4d9e-bae1-afffd826821b admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:18.887382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2051e4-fd0d-4d9e-bae1-afffd826821b admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:18.888082 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2051e4-fd0d-4d9e-bae1-afffd826821b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:18.890101 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2051e4-fd0d-4d9e-bae1-afffd826821b admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:18.980442 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1422/2852] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:18 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:18.984323 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bfbb3-7b21-454d-a6f5-e7f8dc7c854a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:18.994921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bfbb3-7b21-454d-a6f5-e7f8dc7c854a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:18.999609 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fc41c-e843-4b23-a2b3-d8036df241dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:19.000266 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fc41c-e843-4b23-a2b3-d8036df241dd admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:19.000538 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fc41c-e843-4b23-a2b3-d8036df241dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:19.000815 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fc41c-e843-4b23-a2b3-d8036df241dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:19.001504 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fc41c-e843-4b23-a2b3-d8036df241dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:19.003612 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fc41c-e843-4b23-a2b3-d8036df241dd admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:19.013940 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1431/2853] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:19.018530 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2051e4-fd0d-4d9e-bae1-afffd826821b admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:19.028305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2051e4-fd0d-4d9e-bae1-afffd826821b admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:19.032979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34dc8b1-29c2-44a8-adf5-a2c8ca65171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:19.033678 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dc8b1-29c2-44a8-adf5-a2c8ca65171e admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:19.033956 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dc8b1-29c2-44a8-adf5-a2c8ca65171e admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:19.034234 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dc8b1-29c2-44a8-adf5-a2c8ca65171e admin admin] PATH_INFO: `/v3/projects/9ec2f281494642c5a34dd64189056a1e/users/6bce8d30ed1e4e9e9744fb132d14b8b2/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:19.048680 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34dc8b1-29c2-44a8-adf5-a2c8ca65171e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec2f281494642c5a34dd64189056a1e, user_id=6bce8d30ed1e4e9e9744fb132d14b8b2, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:19.051359 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34dc8b1-29c2-44a8-adf5-a2c8ca65171e admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:19.141445 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1423/2854] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:19 2022] PUT /identity/v3/projects/9ec2f281494642c5a34dd64189056a1e/users/6bce8d30ed1e4e9e9744fb132d14b8b2/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:19.146092 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fc41c-e843-4b23-a2b3-d8036df241dd admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:19.156291 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fc41c-e843-4b23-a2b3-d8036df241dd admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:19.175588 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:19.214056 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3152e-c5e4-4810-b90e-bcf14bdf126a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:19.214695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3152e-c5e4-4810-b90e-bcf14bdf126a admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:19.215039 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3152e-c5e4-4810-b90e-bcf14bdf126a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:19.215321 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3152e-c5e4-4810-b90e-bcf14bdf126a admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:19.216047 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3152e-c5e4-4810-b90e-bcf14bdf126a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:19.217701 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3152e-c5e4-4810-b90e-bcf14bdf126a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:19.229121 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1432/2855] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:19.233284 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34dc8b1-29c2-44a8-adf5-a2c8ca65171e admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:19.239183 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:19.244439 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34dc8b1-29c2-44a8-adf5-a2c8ca65171e admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:19.249300 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1cb98-e7f6-47a5-91d3-bcd4ce4877f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:19.250017 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1cb98-e7f6-47a5-91d3-bcd4ce4877f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:19.250302 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1cb98-e7f6-47a5-91d3-bcd4ce4877f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:19.250581 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1cb98-e7f6-47a5-91d3-bcd4ce4877f0 admin admin] PATH_INFO: `/v3/projects/9ec2f281494642c5a34dd64189056a1e/users/6bce8d30ed1e4e9e9744fb132d14b8b2/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:19.253217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1cb98-e7f6-47a5-91d3-bcd4ce4877f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec2f281494642c5a34dd64189056a1e, user_id=6bce8d30ed1e4e9e9744fb132d14b8b2, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:19.255882 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1cb98-e7f6-47a5-91d3-bcd4ce4877f0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:19.265888 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1424/2856] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:19 2022] PUT /identity/v3/projects/9ec2f281494642c5a34dd64189056a1e/users/6bce8d30ed1e4e9e9744fb132d14b8b2/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:19.331833 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd1464-2486-4e68-9a90-dab5e719af82 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:19.332103 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd1464-2486-4e68-9a90-dab5e719af82 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:19.332378 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd1464-2486-4e68-9a90-dab5e719af82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:19.359277 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-25bd1464-2486-4e68-9a90-dab5e719af82 None None] MFA Rules not processed for user `6bce8d30ed1e4e9e9744fb132d14b8b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:19.360517 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-25bd1464-2486-4e68-9a90-dab5e719af82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:19.431801 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1433/2857] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:19.443465 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1cb98-e7f6-47a5-91d3-bcd4ce4877f0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:19.445171 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-22a1cb98-e7f6-47a5-91d3-bcd4ce4877f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:19.503222 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1cb98-e7f6-47a5-91d3-bcd4ce4877f0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:19.506746 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe5f31-7c0e-446d-b0d1-16125db38485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:19.507352 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe5f31-7c0e-446d-b0d1-16125db38485 service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:19.507630 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe5f31-7c0e-446d-b0d1-16125db38485 service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:19.507931 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe5f31-7c0e-446d-b0d1-16125db38485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:19.516548 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe5f31-7c0e-446d-b0d1-16125db38485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:19.518471 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe5f31-7c0e-446d-b0d1-16125db38485 service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:19.527565 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1425/2858] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:19.627751 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd1464-2486-4e68-9a90-dab5e719af82 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:19.637396 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd1464-2486-4e68-9a90-dab5e719af82 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:19.640910 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69470993-7b88-4de1-95cb-42d5d87952f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:19.641510 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470993-7b88-4de1-95cb-42d5d87952f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:19.641786 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470993-7b88-4de1-95cb-42d5d87952f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:19.642057 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470993-7b88-4de1-95cb-42d5d87952f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:19.697639 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69470993-7b88-4de1-95cb-42d5d87952f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:19.699067 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69470993-7b88-4de1-95cb-42d5d87952f4 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:19.708342 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1434/2859] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:22.696988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe5f31-7c0e-446d-b0d1-16125db38485 service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:22.756787 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe5f31-7c0e-446d-b0d1-16125db38485 service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:22.761341 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:22.761975 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:22.762263 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:22.762535 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] PATH_INFO: `/v3/users/fb3ed322b74346718df312e79dc2e7df` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:22.763714 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3ed322b74346718df312e79dc2e7df)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:22.765672 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:22.908104 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3ed322b74346718df312e79dc2e7df'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:22.915205 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3ed322b74346718df312e79dc2e7df'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:22.979937 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3ed322b74346718df312e79dc2e7df'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:22.980551 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3ed322b74346718df312e79dc2e7df'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:22.981155 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3ed322b74346718df312e79dc2e7df'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:22.999318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1426/2860] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:22 2022] DELETE /identity/v3/users/fb3ed322b74346718df312e79dc2e7df => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:23.003759 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ed0c0-954a-4b4a-a70d-1f273aa623b1 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:23.004248 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ed0c0-954a-4b4a-a70d-1f273aa623b1 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:23.004369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ed0c0-954a-4b4a-a70d-1f273aa623b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:23.022522 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-3b2ed0c0-954a-4b4a-a70d-1f273aa623b1 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:23.023707 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3b2ed0c0-954a-4b4a-a70d-1f273aa623b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:23.062214 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=100 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:23.079548 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1435/2861] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:23.089745 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:23.091137 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:23.105299 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:23.148534 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541e042-4bd2-4cb9-8ee7-3aab35920155 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:23.152791 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3352c1-a301-4fd8-bf9a-fde49057ed52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:23.153407 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3352c1-a301-4fd8-bf9a-fde49057ed52 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:23.153689 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3352c1-a301-4fd8-bf9a-fde49057ed52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:23.153963 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3352c1-a301-4fd8-bf9a-fde49057ed52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:23.163320 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3352c1-a301-4fd8-bf9a-fde49057ed52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:23.164730 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3352c1-a301-4fd8-bf9a-fde49057ed52 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:23.174030 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1427/2862] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:23.559174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ed0c0-954a-4b4a-a70d-1f273aa623b1 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:23.560654 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-3b2ed0c0-954a-4b4a-a70d-1f273aa623b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:23.572729 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ed0c0-954a-4b4a-a70d-1f273aa623b1 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:23.579626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:23.579626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:23.579626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:23.579626 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] PATH_INFO: `/v3/projects/7f0456227ea8478dbac663788e6b1dd9` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:23.581577 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f0456227ea8478dbac663788e6b1dd9)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:23.585050 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:23.666204 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f0456227ea8478dbac663788e6b1dd9', 'request_id': 'req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:23.672331 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f0456227ea8478dbac663788e6b1dd9', 'request_id': 'req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:23.678233 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f0456227ea8478dbac663788e6b1dd9', 'request_id': 'req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:23.745733 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] The token cache is being invalidate because project 7f0456227ea8478dbac663788e6b1dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:23.746023 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:23.746647 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:23.748175 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1436/2863] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:23 2022] DELETE /identity/v3/projects/7f0456227ea8478dbac663788e6b1dd9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:23.752170 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3352c1-a301-4fd8-bf9a-fde49057ed52 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:23.761728 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3352c1-a301-4fd8-bf9a-fde49057ed52 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:23.817140 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:23.817785 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:23.818098 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:23.818382 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] PATH_INFO: `/v3/users/6bce8d30ed1e4e9e9744fb132d14b8b2` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:23.819598 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bce8d30ed1e4e9e9744fb132d14b8b2)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:23.821575 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:24.016816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bce8d30ed1e4e9e9744fb132d14b8b2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:24.024172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bce8d30ed1e4e9e9744fb132d14b8b2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:24.083522 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bce8d30ed1e4e9e9744fb132d14b8b2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:24.084186 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bce8d30ed1e4e9e9744fb132d14b8b2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:24.084828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-8cff7249-214e-41fc-ab68-8b91741ad2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bce8d30ed1e4e9e9744fb132d14b8b2'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:24.102823 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1428/2864] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:23 2022] DELETE /identity/v3/users/6bce8d30ed1e4e9e9744fb132d14b8b2 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:24.513721 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:24.515148 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:24.528648 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf0f51-990e-42ea-ad9b-3ccd747c6e91 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:24.585970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:24.586530 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:24.586813 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:24.587482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] PATH_INFO: `/v3/projects/9ec2f281494642c5a34dd64189056a1e` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:24.588346 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec2f281494642c5a34dd64189056a1e)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:24.590324 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:24.683694 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec2f281494642c5a34dd64189056a1e', 'request_id': 'req-07fb9db2-db55-4a97-92b2-2dd76ca6d559'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:24.689722 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec2f281494642c5a34dd64189056a1e', 'request_id': 'req-07fb9db2-db55-4a97-92b2-2dd76ca6d559'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:24.695745 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec2f281494642c5a34dd64189056a1e', 'request_id': 'req-07fb9db2-db55-4a97-92b2-2dd76ca6d559'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:24.763957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:24.768492 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] The token cache is being invalidate because project 9ec2f281494642c5a34dd64189056a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:24.768776 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:24.769419 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-07fb9db2-db55-4a97-92b2-2dd76ca6d559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:24.770949 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1437/2865] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:24 2022] DELETE /identity/v3/projects/9ec2f281494642c5a34dd64189056a1e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:24.776124 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2706a3-b329-4813-810e-171afc882d30 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:24.776606 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2706a3-b329-4813-810e-171afc882d30 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:24.776713 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2706a3-b329-4813-810e-171afc882d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:24.795948 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ba2706a3-b329-4813-810e-171afc882d30 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:24.797190 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ba2706a3-b329-4813-810e-171afc882d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:24.826924 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:24.856295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1429/2866] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:24.864641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb75e-dbc4-4e1b-bee9-c1090479dc1e None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:24.867767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb75e-dbc4-4e1b-bee9-c1090479dc1e None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:24.867767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cb75e-dbc4-4e1b-bee9-c1090479dc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:24.886036 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-305cb75e-dbc4-4e1b-bee9-c1090479dc1e None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:24.887223 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-305cb75e-dbc4-4e1b-bee9-c1090479dc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:24.895211 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1438/2867] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:24.899243 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2706a3-b329-4813-810e-171afc882d30 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:24.908833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2706a3-b329-4813-810e-171afc882d30 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:24.913406 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da7486-34d2-4f07-8f41-ed275d38634a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:24.914020 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da7486-34d2-4f07-8f41-ed275d38634a admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:24.914306 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da7486-34d2-4f07-8f41-ed275d38634a admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:24.914586 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da7486-34d2-4f07-8f41-ed275d38634a admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:24.915283 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da7486-34d2-4f07-8f41-ed275d38634a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:35:24.915594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da7486-34d2-4f07-8f41-ed275d38634a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:24.917272 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da7486-34d2-4f07-8f41-ed275d38634a admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:24.928896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1430/2868] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:35:24.932821 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb75e-dbc4-4e1b-bee9-c1090479dc1e None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:24.942599 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cb75e-dbc4-4e1b-bee9-c1090479dc1e None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:24.947247 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0e54e-cdad-4c72-9edf-95e7d6e8a266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:24.947871 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e54e-cdad-4c72-9edf-95e7d6e8a266 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:24.948158 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e54e-cdad-4c72-9edf-95e7d6e8a266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:24.948435 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e54e-cdad-4c72-9edf-95e7d6e8a266 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:24.949111 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0e54e-cdad-4c72-9edf-95e7d6e8a266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:24.950977 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0e54e-cdad-4c72-9edf-95e7d6e8a266 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:25.045253 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1439/2869] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:24 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:25.050666 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da7486-34d2-4f07-8f41-ed275d38634a admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:25.060811 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da7486-34d2-4f07-8f41-ed275d38634a admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:25.122635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a7085-d32a-4a96-b9d9-32f3a050c7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:25.123181 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a7085-d32a-4a96-b9d9-32f3a050c7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:25.123472 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a7085-d32a-4a96-b9d9-32f3a050c7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:25.123747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a7085-d32a-4a96-b9d9-32f3a050c7f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:25.124456 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a7085-d32a-4a96-b9d9-32f3a050c7f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:25.126487 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a7085-d32a-4a96-b9d9-32f3a050c7f8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:25.209290 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1431/2870] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:25 2022] POST /identity/v3/users => generated 284 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:35:25.213602 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0e54e-cdad-4c72-9edf-95e7d6e8a266 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:25.224292 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0e54e-cdad-4c72-9edf-95e7d6e8a266 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:25.228897 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4da04-98c2-4853-881c-7b2786c381d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:25.229502 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4da04-98c2-4853-881c-7b2786c381d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:25.229785 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4da04-98c2-4853-881c-7b2786c381d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:25.230063 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4da04-98c2-4853-881c-7b2786c381d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:25.230794 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4da04-98c2-4853-881c-7b2786c381d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:25.232474 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4da04-98c2-4853-881c-7b2786c381d6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:25.243682 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1440/2871] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:25.247389 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a7085-d32a-4a96-b9d9-32f3a050c7f8 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:25.257228 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a7085-d32a-4a96-b9d9-32f3a050c7f8 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:25.261778 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7920e-400a-486d-936d-223f040313b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:25.262374 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7920e-400a-486d-936d-223f040313b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:25.262659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7920e-400a-486d-936d-223f040313b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:25.262938 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7920e-400a-486d-936d-223f040313b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:25.263621 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7920e-400a-486d-936d-223f040313b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:25.265267 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7920e-400a-486d-936d-223f040313b9 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:25.275593 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1432/2872] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:25.279501 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4da04-98c2-4853-881c-7b2786c381d6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:25.289753 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4da04-98c2-4853-881c-7b2786c381d6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:25.294319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca248ab1-a9c4-49ac-8492-ab8ef6b819b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:25.294967 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca248ab1-a9c4-49ac-8492-ab8ef6b819b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:25.295252 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca248ab1-a9c4-49ac-8492-ab8ef6b819b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:25.295540 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca248ab1-a9c4-49ac-8492-ab8ef6b819b6 admin admin] PATH_INFO: `/v3/projects/ea060a21574a4d59a7571cee5df637cf/users/df5dad5e8fb34036950912d5e74d8e5f/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:25.310316 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca248ab1-a9c4-49ac-8492-ab8ef6b819b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea060a21574a4d59a7571cee5df637cf, user_id=df5dad5e8fb34036950912d5e74d8e5f, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:25.313010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca248ab1-a9c4-49ac-8492-ab8ef6b819b6 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:25.512615 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1441/2873] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:25 2022] PUT /identity/v3/projects/ea060a21574a4d59a7571cee5df637cf/users/df5dad5e8fb34036950912d5e74d8e5f/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:35:25.523378 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d0813-81f6-4926-a014-fbcb1dd61597 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:25.523667 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d0813-81f6-4926-a014-fbcb1dd61597 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:25.523979 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d0813-81f6-4926-a014-fbcb1dd61597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:25.550919 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-e40d0813-81f6-4926-a014-fbcb1dd61597 None None] MFA Rules not processed for user `df5dad5e8fb34036950912d5e74d8e5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:25.552179 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-e40d0813-81f6-4926-a014-fbcb1dd61597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:25.625295 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1433/2874] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:25.629898 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca248ab1-a9c4-49ac-8492-ab8ef6b819b6 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:25.696518 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca248ab1-a9c4-49ac-8492-ab8ef6b819b6 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:25.701142 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e195e-d64b-46ec-8988-36a9a63ece03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:25.701778 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e195e-d64b-46ec-8988-36a9a63ece03 admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:25.702041 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e195e-d64b-46ec-8988-36a9a63ece03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:25.702319 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e195e-d64b-46ec-8988-36a9a63ece03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:25.703008 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e195e-d64b-46ec-8988-36a9a63ece03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:25.704929 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e195e-d64b-46ec-8988-36a9a63ece03 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:25.778349 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1442/2875] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:25 2022] POST /identity/v3/projects => generated 367 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:25.782335 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d0813-81f6-4926-a014-fbcb1dd61597 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:25.793486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d0813-81f6-4926-a014-fbcb1dd61597 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:25.820331 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:25.849997 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15b404-a4d7-423d-a8c8-e17f98b8af29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:25.850706 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15b404-a4d7-423d-a8c8-e17f98b8af29 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:25.850988 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15b404-a4d7-423d-a8c8-e17f98b8af29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:25.851265 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15b404-a4d7-423d-a8c8-e17f98b8af29 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:25.852000 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15b404-a4d7-423d-a8c8-e17f98b8af29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:25.853957 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15b404-a4d7-423d-a8c8-e17f98b8af29 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:25.963173 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1434/2876] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:25 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 01:35:25.967217 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e195e-d64b-46ec-8988-36a9a63ece03 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:25.977568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e195e-d64b-46ec-8988-36a9a63ece03 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:25.982128 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d44dca-8a48-4855-ab5d-a9ea1d394989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:25.982736 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d44dca-8a48-4855-ab5d-a9ea1d394989 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:25.983049 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d44dca-8a48-4855-ab5d-a9ea1d394989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:25.983329 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d44dca-8a48-4855-ab5d-a9ea1d394989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:25.984060 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d44dca-8a48-4855-ab5d-a9ea1d394989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:25.985676 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d44dca-8a48-4855-ab5d-a9ea1d394989 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:25.996684 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1443/2877] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:26.000809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15b404-a4d7-423d-a8c8-e17f98b8af29 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:26.011277 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15b404-a4d7-423d-a8c8-e17f98b8af29 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:26.015862 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44100cb3-31b9-4240-bab6-b44ed9d41898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:26.016518 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44100cb3-31b9-4240-bab6-b44ed9d41898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:26.016796 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44100cb3-31b9-4240-bab6-b44ed9d41898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:26.017072 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44100cb3-31b9-4240-bab6-b44ed9d41898 admin admin] PATH_INFO: `/v3/projects/480ad435a19d4c4d81701337fc5a31ef/users/08e64a381c534393915eccce2dca59ed/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:26.032044 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44100cb3-31b9-4240-bab6-b44ed9d41898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480ad435a19d4c4d81701337fc5a31ef, user_id=08e64a381c534393915eccce2dca59ed, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:26.034653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44100cb3-31b9-4240-bab6-b44ed9d41898 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:26.108172 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1435/2878] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:25 2022] PUT /identity/v3/projects/480ad435a19d4c4d81701337fc5a31ef/users/08e64a381c534393915eccce2dca59ed/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 01:35:26.112680 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d44dca-8a48-4855-ab5d-a9ea1d394989 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:26.122919 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d44dca-8a48-4855-ab5d-a9ea1d394989 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:26.177865 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084046f-bc78-4262-9ad8-c6b772edaf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:26.178482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084046f-bc78-4262-9ad8-c6b772edaf50 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:26.178767 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084046f-bc78-4262-9ad8-c6b772edaf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:26.179043 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084046f-bc78-4262-9ad8-c6b772edaf50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:26.179758 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084046f-bc78-4262-9ad8-c6b772edaf50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:26.181394 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084046f-bc78-4262-9ad8-c6b772edaf50 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:26.191442 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1444/2879] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:26.195726 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44100cb3-31b9-4240-bab6-b44ed9d41898 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:26.205845 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44100cb3-31b9-4240-bab6-b44ed9d41898 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:26.210653 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91113a4-efd6-438c-859d-b8d732a91a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:26.211308 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91113a4-efd6-438c-859d-b8d732a91a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:26.211585 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91113a4-efd6-438c-859d-b8d732a91a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:26.211890 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91113a4-efd6-438c-859d-b8d732a91a11 admin admin] PATH_INFO: `/v3/projects/480ad435a19d4c4d81701337fc5a31ef/users/08e64a381c534393915eccce2dca59ed/roles/933d8f1f9c734bb7beb89f92643c9f56` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:26.213792 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91113a4-efd6-438c-859d-b8d732a91a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480ad435a19d4c4d81701337fc5a31ef, user_id=08e64a381c534393915eccce2dca59ed, role_id=933d8f1f9c734bb7beb89f92643c9f56)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:26.216461 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91113a4-efd6-438c-859d-b8d732a91a11 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:26.226594 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1436/2880] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:26 2022] PUT /identity/v3/projects/480ad435a19d4c4d81701337fc5a31ef/users/08e64a381c534393915eccce2dca59ed/roles/933d8f1f9c734bb7beb89f92643c9f56 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:26.238369 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c08360-a6b2-4100-9f17-19d56dda72f5 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:26.238702 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c08360-a6b2-4100-9f17-19d56dda72f5 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:26.239222 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c08360-a6b2-4100-9f17-19d56dda72f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:26.247251 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:26.267236 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-a7c08360-a6b2-4100-9f17-19d56dda72f5 None None] MFA Rules not processed for user `08e64a381c534393915eccce2dca59ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:26.268493 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-a7c08360-a6b2-4100-9f17-19d56dda72f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:26.334224 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1445/2881] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:26.345178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91113a4-efd6-438c-859d-b8d732a91a11 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:26.346901 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-f91113a4-efd6-438c-859d-b8d732a91a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:26.403076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91113a4-efd6-438c-859d-b8d732a91a11 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:26.406747 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de22b1e-2a6b-4292-be6b-f36584eed6cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:26.407370 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de22b1e-2a6b-4292-be6b-f36584eed6cc service cinder] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:26.407656 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de22b1e-2a6b-4292-be6b-f36584eed6cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:26.407970 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de22b1e-2a6b-4292-be6b-f36584eed6cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:26.417071 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de22b1e-2a6b-4292-be6b-f36584eed6cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:26.418536 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de22b1e-2a6b-4292-be6b-f36584eed6cc service cinder] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:26.429876 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1437/2882] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:26.646749 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c08360-a6b2-4100-9f17-19d56dda72f5 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:26.656568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c08360-a6b2-4100-9f17-19d56dda72f5 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:26.660741 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a24052-f41d-4ac7-b103-81f9028b6a24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:26.661481 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a24052-f41d-4ac7-b103-81f9028b6a24 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:26.661603 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a24052-f41d-4ac7-b103-81f9028b6a24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:26.661879 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a24052-f41d-4ac7-b103-81f9028b6a24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:26.723640 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a24052-f41d-4ac7-b103-81f9028b6a24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:26.725126 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a24052-f41d-4ac7-b103-81f9028b6a24 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:26.734413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1446/2883] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:33.097021 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de22b1e-2a6b-4292-be6b-f36584eed6cc service cinder] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:33.158149 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de22b1e-2a6b-4292-be6b-f36584eed6cc service cinder] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:33.162684 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:33.163318 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:33.163597 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:33.163905 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] PATH_INFO: `/v3/users/df5dad5e8fb34036950912d5e74d8e5f` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:33.165039 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df5dad5e8fb34036950912d5e74d8e5f)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:33.166955 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:33.243909 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:33.319659 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df5dad5e8fb34036950912d5e74d8e5f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:33.326467 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df5dad5e8fb34036950912d5e74d8e5f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:33.449567 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df5dad5e8fb34036950912d5e74d8e5f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:33.450259 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df5dad5e8fb34036950912d5e74d8e5f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:33.450858 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df5dad5e8fb34036950912d5e74d8e5f'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:33.469321 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1438/2884] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:33 2022] DELETE /identity/v3/users/df5dad5e8fb34036950912d5e74d8e5f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:33.473820 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59f5e4-89c9-4b7d-9e13-d71a29b69f2c None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:33.474336 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59f5e4-89c9-4b7d-9e13-d71a29b69f2c None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:33.474462 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59f5e4-89c9-4b7d-9e13-d71a29b69f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:33.495141 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-da59f5e4-89c9-4b7d-9e13-d71a29b69f2c None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:33.496408 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-da59f5e4-89c9-4b7d-9e13-d71a29b69f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:33.554564 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1447/2885] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 01:35:33.564178 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:33.566110 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:33.623398 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd0063-d58e-4dec-ae25-7d6771724367 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:33.626949 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021b8c8-abf8-4d31-85fa-ee09157a9a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:33.628194 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021b8c8-abf8-4d31-85fa-ee09157a9a3a service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:33.628477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021b8c8-abf8-4d31-85fa-ee09157a9a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:33.628748 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021b8c8-abf8-4d31-85fa-ee09157a9a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:33.637803 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021b8c8-abf8-4d31-85fa-ee09157a9a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:33.639244 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021b8c8-abf8-4d31-85fa-ee09157a9a3a service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:33.648711 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1439/2886] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:34.051568 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59f5e4-89c9-4b7d-9e13-d71a29b69f2c None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:34.053138 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-da59f5e4-89c9-4b7d-9e13-d71a29b69f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:34.065921 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59f5e4-89c9-4b7d-9e13-d71a29b69f2c None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:34.069987 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:34.070641 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:34.070937 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:34.071216 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] PATH_INFO: `/v3/projects/ea060a21574a4d59a7571cee5df637cf` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:34.072447 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea060a21574a4d59a7571cee5df637cf)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:34.074362 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:34.099511 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:34.227096 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea060a21574a4d59a7571cee5df637cf', 'request_id': 'req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:34.233237 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea060a21574a4d59a7571cee5df637cf', 'request_id': 'req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:34.239397 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea060a21574a4d59a7571cee5df637cf', 'request_id': 'req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:34.345181 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] The token cache is being invalidate because project ea060a21574a4d59a7571cee5df637cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:34.345466 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:34.346081 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:34.348197 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1448/2887] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:34 2022] DELETE /identity/v3/projects/ea060a21574a4d59a7571cee5df637cf => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:34.352158 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021b8c8-abf8-4d31-85fa-ee09157a9a3a service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:34.361870 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021b8c8-abf8-4d31-85fa-ee09157a9a3a service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:34.416526 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:34.417162 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:34.417443 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:34.417717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] PATH_INFO: `/v3/users/08e64a381c534393915eccce2dca59ed` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:34.418874 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e64a381c534393915eccce2dca59ed)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:34.420896 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:34.610841 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e64a381c534393915eccce2dca59ed'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:34.612060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:34.618592 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e64a381c534393915eccce2dca59ed'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:34.712520 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e64a381c534393915eccce2dca59ed'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:34.712658 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e64a381c534393915eccce2dca59ed'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:34.713248 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-023d9054-a980-437f-9007-977cd8c860c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e64a381c534393915eccce2dca59ed'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:34.730677 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1440/2888] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:34 2022] DELETE /identity/v3/users/08e64a381c534393915eccce2dca59ed => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:35.114439 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:35.116040 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:35.132283 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f2dc1-0a66-4bc7-9b7e-63651f2f3e84 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:35.195731 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:35.196395 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:35.196672 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:35.196944 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] PATH_INFO: `/v3/projects/480ad435a19d4c4d81701337fc5a31ef` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:35.198079 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=480ad435a19d4c4d81701337fc5a31ef)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:35.200010 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:35.294275 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480ad435a19d4c4d81701337fc5a31ef', 'request_id': 'req-de3ca549-03d2-4bb5-983b-54ec92350011'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:35.300909 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480ad435a19d4c4d81701337fc5a31ef', 'request_id': 'req-de3ca549-03d2-4bb5-983b-54ec92350011'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:35.306301 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480ad435a19d4c4d81701337fc5a31ef', 'request_id': 'req-de3ca549-03d2-4bb5-983b-54ec92350011'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:35.368941 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] The token cache is being invalidate because project 480ad435a19d4c4d81701337fc5a31ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:35.369230 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:35.369865 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-de3ca549-03d2-4bb5-983b-54ec92350011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:35.371399 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1449/2889] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:35 2022] DELETE /identity/v3/projects/480ad435a19d4c4d81701337fc5a31ef => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:35.376217 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59fd33-7e33-46c7-a4e9-2af1dacdd9c2 None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:35.376719 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59fd33-7e33-46c7-a4e9-2af1dacdd9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:35.376833 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59fd33-7e33-46c7-a4e9-2af1dacdd9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:35.396469 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-ca59fd33-7e33-46c7-a4e9-2af1dacdd9c2 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:35.397749 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-ca59fd33-7e33-46c7-a4e9-2af1dacdd9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:35.455809 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1441/2890] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:35.463339 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f41464-e3cd-43aa-b881-e0de9399d304 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:35.463625 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f41464-e3cd-43aa-b881-e0de9399d304 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:35.463946 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f41464-e3cd-43aa-b881-e0de9399d304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:35.483545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-58f41464-e3cd-43aa-b881-e0de9399d304 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:35.484779 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-58f41464-e3cd-43aa-b881-e0de9399d304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:35.494014 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1450/2891] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:35.498198 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59fd33-7e33-46c7-a4e9-2af1dacdd9c2 None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:35.508945 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59fd33-7e33-46c7-a4e9-2af1dacdd9c2 None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:35.513686 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f479222-843d-4cc9-b272-31edecdef9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:35.514294 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f479222-843d-4cc9-b272-31edecdef9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:35.514643 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f479222-843d-4cc9-b272-31edecdef9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:35.515002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f479222-843d-4cc9-b272-31edecdef9c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:35.515883 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f479222-843d-4cc9-b272-31edecdef9c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80524) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 01:35:35.516191 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f479222-843d-4cc9-b272-31edecdef9c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:35.517863 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f479222-843d-4cc9-b272-31edecdef9c3 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:35.531176 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1442/2892] 10.222.0.25 () {58 vars in 1153 bytes} [Sat Aug 6 01:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 01:35:35.538303 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f41464-e3cd-43aa-b881-e0de9399d304 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:35.547482 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f41464-e3cd-43aa-b881-e0de9399d304 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:35.552807 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f29af-ac5c-43a8-926c-acd4b924ea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:35.553413 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f29af-ac5c-43a8-926c-acd4b924ea8b admin admin] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:35.553695 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f29af-ac5c-43a8-926c-acd4b924ea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:35.553970 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f29af-ac5c-43a8-926c-acd4b924ea8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:35.554642 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f29af-ac5c-43a8-926c-acd4b924ea8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:35.556597 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f29af-ac5c-43a8-926c-acd4b924ea8b admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:35.684996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1451/2893] 10.222.0.25 () {60 vars in 1153 bytes} [Sat Aug 6 01:35:35 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:35.689079 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f479222-843d-4cc9-b272-31edecdef9c3 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:35.699002 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f479222-843d-4cc9-b272-31edecdef9c3 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:35.755422 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedf564-3adf-4626-a626-93b728de14c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:35.756013 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedf564-3adf-4626-a626-93b728de14c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:35.756297 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedf564-3adf-4626-a626-93b728de14c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:35.756583 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedf564-3adf-4626-a626-93b728de14c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:35.757250 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedf564-3adf-4626-a626-93b728de14c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:35.759167 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedf564-3adf-4626-a626-93b728de14c0 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:35.850381 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1443/2894] 10.222.0.25 () {60 vars in 1144 bytes} [Sat Aug 6 01:35:35 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 01:35:35.854452 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f29af-ac5c-43a8-926c-acd4b924ea8b admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:35.864903 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f29af-ac5c-43a8-926c-acd4b924ea8b admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:35.869514 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba43be-b3cf-4f8b-8928-09a9f15721b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:35.871535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba43be-b3cf-4f8b-8928-09a9f15721b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:35.871535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba43be-b3cf-4f8b-8928-09a9f15721b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:35.871535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba43be-b3cf-4f8b-8928-09a9f15721b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:35.871535 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba43be-b3cf-4f8b-8928-09a9f15721b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:35.872911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba43be-b3cf-4f8b-8928-09a9f15721b1 admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:35.882775 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1452/2895] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:35.886545 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedf564-3adf-4626-a626-93b728de14c0 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:35.896943 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedf564-3adf-4626-a626-93b728de14c0 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:35.901479 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76e8ff-99bc-43da-9a64-ee63aa4b39e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:35.902076 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76e8ff-99bc-43da-9a64-ee63aa4b39e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:35.902372 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76e8ff-99bc-43da-9a64-ee63aa4b39e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:35.902629 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76e8ff-99bc-43da-9a64-ee63aa4b39e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:35.903305 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76e8ff-99bc-43da-9a64-ee63aa4b39e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:35.904914 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76e8ff-99bc-43da-9a64-ee63aa4b39e8 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:35.915836 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1444/2896] 10.222.0.25 () {58 vars in 1122 bytes} [Sat Aug 6 01:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 01:35:35.919698 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba43be-b3cf-4f8b-8928-09a9f15721b1 admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:35.924878 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:35.930104 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba43be-b3cf-4f8b-8928-09a9f15721b1 admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:35.934713 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fac4fd-1bd4-4c97-a5fe-4e0aac7edd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:35.935360 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fac4fd-1bd4-4c97-a5fe-4e0aac7edd6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:35.935646 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fac4fd-1bd4-4c97-a5fe-4e0aac7edd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:35.935957 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fac4fd-1bd4-4c97-a5fe-4e0aac7edd6a admin admin] PATH_INFO: `/v3/projects/bd0ef87dc64943dc8a3208ae9cec0c3d/users/cec2e8d9faf14d2e9a66e6b30bac6dd1/roles/512a743b8ca748f8b793681748be7906` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:35.950140 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fac4fd-1bd4-4c97-a5fe-4e0aac7edd6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0ef87dc64943dc8a3208ae9cec0c3d, user_id=cec2e8d9faf14d2e9a66e6b30bac6dd1, role_id=512a743b8ca748f8b793681748be7906)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:35.952844 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fac4fd-1bd4-4c97-a5fe-4e0aac7edd6a admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:36.003120 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1453/2897] 10.222.0.25 () {60 vars in 1483 bytes} [Sat Aug 6 01:35:35 2022] PUT /identity/v3/projects/bd0ef87dc64943dc8a3208ae9cec0c3d/users/cec2e8d9faf14d2e9a66e6b30bac6dd1/roles/512a743b8ca748f8b793681748be7906 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:36.014830 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736f17c-9a1b-42c1-b40f-4e6ad03f91ab None None] REQUEST_METHOD: `POST` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:36.015075 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736f17c-9a1b-42c1-b40f-4e6ad03f91ab None None] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:36.015332 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736f17c-9a1b-42c1-b40f-4e6ad03f91ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:36.043169 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.auth.core [None req-c736f17c-9a1b-42c1-b40f-4e6ad03f91ab None None] MFA Rules not processed for user `cec2e8d9faf14d2e9a66e6b30bac6dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80524) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:36.044454 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-c736f17c-9a1b-42c1-b40f-4e6ad03f91ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:36.062651 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:36.109828 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1445/2898] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:36.120913 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fac4fd-1bd4-4c97-a5fe-4e0aac7edd6a admin admin] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:36.122712 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-c4fac4fd-1bd4-4c97-a5fe-4e0aac7edd6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:36.182058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fac4fd-1bd4-4c97-a5fe-4e0aac7edd6a admin admin] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:36.188545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6409377-d9a8-47b9-ba50-00b1c058e2c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd0e588d52e40f08a2dc1062ecd4232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:36.188545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6409377-d9a8-47b9-ba50-00b1c058e2c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:36.188545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6409377-d9a8-47b9-ba50-00b1c058e2c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:36.188545 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6409377-d9a8-47b9-ba50-00b1c058e2c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:36.195754 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6409377-d9a8-47b9-ba50-00b1c058e2c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:36.197317 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6409377-d9a8-47b9-ba50-00b1c058e2c9 service cinder] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:36.206312 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1454/2899] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:40.023491 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736f17c-9a1b-42c1-b40f-4e6ad03f91ab None None] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:40.083537 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736f17c-9a1b-42c1-b40f-4e6ad03f91ab None None] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:40.088050 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:40.088717 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:40.088999 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:40.089275 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] PATH_INFO: `/v3/users/cec2e8d9faf14d2e9a66e6b30bac6dd1` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:40.090418 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec2e8d9faf14d2e9a66e6b30bac6dd1)` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:40.092375 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:40.267353 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec2e8d9faf14d2e9a66e6b30bac6dd1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:40.274069 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec2e8d9faf14d2e9a66e6b30bac6dd1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:40.362477 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec2e8d9faf14d2e9a66e6b30bac6dd1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:40.363108 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec2e8d9faf14d2e9a66e6b30bac6dd1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:40.363739 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.notifications [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec2e8d9faf14d2e9a66e6b30bac6dd1'} {{(pid=80524) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:40.381109 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1446/2900] 10.222.0.25 () {58 vars in 1224 bytes} [Sat Aug 6 01:35:40 2022] DELETE /identity/v3/users/cec2e8d9faf14d2e9a66e6b30bac6dd1 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:40.385575 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa34c77-5fed-4764-a50a-7605b58415a9 None None] REQUEST_METHOD: `POST` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:40.386058 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa34c77-5fed-4764-a50a-7605b58415a9 None None] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:40.386174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa34c77-5fed-4764-a50a-7605b58415a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:40.405153 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.auth.core [None req-5aa34c77-5fed-4764-a50a-7605b58415a9 None None] MFA Rules not processed for user `bb37d398d5fd456aa46fd2b5f5576815`. Rule list: `[]` (Enabled: `True`). {{(pid=80525) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 01:35:40.406972 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5aa34c77-5fed-4764-a50a-7605b58415a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:40.463174 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1455/2901] 10.222.0.25 () {58 vars in 958 bytes} [Sat Aug 6 01:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 01:35:40.473676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:40.475060 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:40.516987 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:40.531025 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b156d6-85ed-4815-a5e8-a08928abb308 admin admin] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:40.534676 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb6552-2780-449a-b714-dc82c35ed468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c14fb133a0f419390fb0d00c600bdd4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:40.535954 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6552-2780-449a-b714-dc82c35ed468 service neutron] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:40.536239 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6552-2780-449a-b714-dc82c35ed468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:40.536516 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6552-2780-449a-b714-dc82c35ed468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:40.545483 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb6552-2780-449a-b714-dc82c35ed468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:40.547816 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb6552-2780-449a-b714-dc82c35ed468 service neutron] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:40.557238 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1447/2902] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:35:41.010406 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa34c77-5fed-4764-a50a-7605b58415a9 None None] Authenticating user token {{(pid=80525) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:35:41.011935 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.fernet_utils [None req-5aa34c77-5fed-4764-a50a-7605b58415a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80525) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:35:41.024596 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa34c77-5fed-4764-a50a-7605b58415a9 None None] Validating token access rules against request {{(pid=80525) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:35:41.029261 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb37d398d5fd456aa46fd2b5f5576815', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '58e81a1daacf4bb08ea62cf133cf1073', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80525) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:35:41.029911 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:35:41.030196 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:35:41.030563 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] PATH_INFO: `/v3/projects/bd0ef87dc64943dc8a3208ae9cec0c3d` {{(pid=80525) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:35:41.031705 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd0ef87dc64943dc8a3208ae9cec0c3d)` {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:35:41.033717 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] RBAC: Authorization granted {{(pid=80525) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:35:41.141086 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd0ef87dc64943dc8a3208ae9cec0c3d', 'request_id': 'req-ecf38ad3-0f79-4493-b573-e82aa1000a8e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:41.148163 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd0ef87dc64943dc8a3208ae9cec0c3d', 'request_id': 'req-ecf38ad3-0f79-4493-b573-e82aa1000a8e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:41.153569 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd0ef87dc64943dc8a3208ae9cec0c3d', 'request_id': 'req-ecf38ad3-0f79-4493-b573-e82aa1000a8e'} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:41.226699 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] The token cache is being invalidate because project bd0ef87dc64943dc8a3208ae9cec0c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80525) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 01:35:41.226996 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:41.227635 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG keystone.notifications [None req-ecf38ad3-0f79-4493-b573-e82aa1000a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80525) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 01:35:41.229182 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: [pid: 80525|app: 0|req: 1456/2903] 10.222.0.25 () {58 vars in 1233 bytes} [Sat Aug 6 01:35:41 2022] DELETE /identity/v3/projects/bd0ef87dc64943dc8a3208ae9cec0c3d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 01:35:50.608913 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=8 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:35:51.222818 dv-cinder-851996-2-iscsi devstack@keystone.service[80525]: DEBUG dbcounter [-] [80525] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 01:36:48.354495 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb6552-2780-449a-b714-dc82c35ed468 service neutron] Authenticating user token {{(pid=80524) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 01:36:48.357247 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-4dfb6552-2780-449a-b714-dc82c35ed468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:48.427546 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb6552-2780-449a-b714-dc82c35ed468 service neutron] Validating token access rules against request {{(pid=80524) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 01:36:48.484457 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc7226-d2ca-41de-a41a-37b1e47dfec7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8361f2f9c70b4a3288a187cb2d97d415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b00f8bff707b4491a724e8aaac10c993', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80524) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 01:36:48.485077 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc7226-d2ca-41de-a41a-37b1e47dfec7 service placement] REQUEST_METHOD: `GET` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 01:36:48.485360 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc7226-d2ca-41de-a41a-37b1e47dfec7 service placement] SCRIPT_NAME: `/identity` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 01:36:48.485635 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc7226-d2ca-41de-a41a-37b1e47dfec7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80524) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 01:36:48.487486 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.fernet_utils [None req-d1bc7226-d2ca-41de-a41a-37b1e47dfec7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80524) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 01:36:48.499631 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc7226-d2ca-41de-a41a-37b1e47dfec7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 01:36:48.501235 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc7226-d2ca-41de-a41a-37b1e47dfec7 service placement] RBAC: Authorization granted {{(pid=80524) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 01:36:48.511038 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: [pid: 80524|app: 0|req: 1448/2904] 10.222.0.25 () {60 vars in 1365 bytes} [Sat Aug 6 01:36:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 01:36:58.506768 dv-cinder-851996-2-iscsi devstack@keystone.service[80524]: DEBUG dbcounter [-] [80524] Writing DB stats keystone:SELECT=58 {{(pid=80524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}